Search Results: "ana"

13 January 2015

Simon Josefsson: Replicant 4.2 0003 on I9300

The Replicant project released version 4.2 0003 recently. I have been using Replicant on a Samsung SIII (I9300) for around 14 months now. Since I have blogged about issues with NFC and Wifi earlier, I wanted to give a status update after upgrading to 0003. I m happy to report that my NFC issue has been resolved in 0003 (the way I suggested; reverting the patch). My issues with Wifi has been improved in 0003, with my merge request being accepted. What follows below is a standalone explanation of what works and what doesn t, as a superset of similar things discussed in my earlier blog posts. What works out of the box: Audio, Telephony, SMS, Data (GSM/3G), Back Camera, NFC. 2D Graphics is somewhat slow compared to stock ROM, but I m using it daily and can live with that so it isn t too onerus. Stability is fine, similar to other Android device I m used to. Video playback does not work (due to non-free media decoders?), which is not a serious problem for me but still likely the biggest outstanding issue except for freedom concerns. 3D graphics apparently doesn t work, and I believe it is what prevents Firefox from working properly (it crashes). I m having one annoying but strange problem with telephony: when calling one person I get scrambled audio around 75% of the time. I can still hear what the other person is saying, but can barely make anything out of it. This only happens over 3G, so my workaround when calling that person is to switch to 2G before and switch back after. I talk with plenty other people, and have never had this problem with anyone else, and it has never happened when she talks with anyone else but me. If anyone has suggestion on how to debug this, I m all ears. Important apps to get through daily life for me includes K9Mail (email), DAVDroid (for ownCloud CalDav/CardDAV), CalDav Sync Adapter (for Google Calendars), Conversations (XMPP/Jabber chat), FDroid (for apps), ownCloud (auto-uploading my photos), SMS Backup+, Xabber (different XMPP/Jabber accounts), Yubico Authenticator, MuPDF and oandbackup. A couple of other apps I find useful are AdAway (remove web ads), AndStatus, Calendar Widget, NewsBlur and ownCloud News Reader (RSS readers), Tinfoil for Facebook, Twidere (I find its UI somewhat nicer than AndStatus s), and c:geo. A number of things requires non-free components. As I discussed in my initial writeup from when I started using Replicant I don t like this, but I m accepting it temporarily. The list of issues that can be fixed by adding non-free components include the front camera, Bluetooth, GPS, and Wifi. After flashing the Replicant ROM image that I built (using the fine build instructions), I m using the following script to add the missing non-free files from Cyanogenmod. 
# Download Cyanogenmod 10.1.3 (Android 4.2-based) binaries:
# wget http://download.cyanogenmod.org/get/jenkins/42508/cm-10.1.3-i9300.zip
# echo "073a464a9f5129c490502c77374495c38a25ba790c10e27f51b43845baeba6bf  cm-10.1.3-i9300.zip"   sha256sum -c 
# unzip cm-10.1.3-i9300.zip
adb root
adb remount
adb shell mkdir /system/vendor/firmware
adb shell chmod 755 /system/vendor/firmware
# Front Camera
adb push cm-10.1.3-i9300/system/vendor/firmware/fimc_is_fw.bin /system/vendor/firmware/fimc_is_fw.bin
adb push cm-10.1.3-i9300/system/vendor/firmware/setfile.bin /system/vendor/firmware/setfile.bin
adb shell chmod 644 /system/vendor/firmware/fimc_is_fw.bin /system/vendor/firmware/setfile.bin
# Bluetooth
adb push cm-10.1.3-i9300/system/bin/bcm4334.hcd /system/vendor/firmware/
adb shell chmod 644 /system/vendor/firmware/bcm4334*.hcd
# GPS
adb push cm-10.1.3-i9300/system/bin/gpsd /system/bin/gpsd
adb shell chmod 755 /system/bin/gpsd
adb push cm-10.1.3-i9300/system/lib/hw/gps.exynos4.so /system/lib/hw/gps.exynos4.so
adb push cm-10.1.3-i9300/system/lib/libsecril-client.so /system/lib/libsecril-client.so
adb shell chmod 644 /system/lib/hw/gps.exynos4.so /system/lib/libsecril-client.so
# Wifi
adb push cm-10.1.3-i9300/system/etc/wifi/bcmdhd_apsta.bin_b1 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/bcmdhd_apsta.bin_b2 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/bcmdhd_mfg.bin_b0 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/bcmdhd_mfg.bin_b1 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/bcmdhd_mfg.bin_b2 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/bcmdhd_p2p.bin_b0 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/bcmdhd_p2p.bin_b1 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/bcmdhd_p2p.bin_b2 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/bcmdhd_sta.bin_b0 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/bcmdhd_sta.bin_b1 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/bcmdhd_sta.bin_b2 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/nvram_mfg.txt /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/nvram_mfg.txt_murata /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/nvram_mfg.txt_murata_b2 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/nvram_mfg.txt_semcosh /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/nvram_net.txt /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/nvram_net.txt_murata /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/nvram_net.txt_murata_b2 /system/vendor/firmware/
adb push cm-10.1.3-i9300/system/etc/wifi/nvram_net.txt_semcosh /system/vendor/firmware/
I hope this helps others switch to a better phone environment!

Daniel Pocock: Silent data loss exposed

I was moving a large number of image files around and decided to compare checksums after putting them in their new home. Ouf of several thousand files, about 80GB of data, I found that seventeen of them had a checksum mismatch. Running md5sum manually on each of those was showing a correct checksum, well, up until the sixth file and then I found this: 
$ md5sum DSC_2624.NEF
94fc8d3cdea3b0f3479fa255f7634b5b  DSC_2624.NEF
$ md5sum DSC_2624.NEF
25cf4469f44ae5e5d6a13c8e2fb220bf  DSC_2624.NEF
$ md5sum DSC_2624.NEF
03a68230b2c6d29a9888d2358ed8e225  DSC_2624.NEF
Yes, each time I run md5sum on the same file it gives a different result. Out of the seventeen files, I found one other displaying the same problem and the others gave correct checksums when I ran md5sum manually. Definitely not a healthy disk, or is it? This is the reason why checksumming filesystems like Btrfs are so important. There are no errors or warnings in the logs on the system with this disk. Silent data loss at its best. Is the disk to blame though? It may be tempting to think this is a disk fault, most people have seen faulty disks at some time or another. In the old days you could often hear them too. There is another possible explanation though: memory corruption. The data read from disk is normally cached in RAM and if the RAM is corrupt, the cache would return bad data. I dropped the read cache: 
# echo 3 > /proc/sys/vm/drop_caches
and tried md5sum again and observed the file checksum is now correct. It would appear the md5sum command had been operating on data in the page cache and the root cause of the problem is memory corruption. Time to run a memory test and then replace the RAM in the machine.

Jonathan McDowell: Tracking a ship around the world

I moved back from the California Bay Area to Belfast a while back and for various reasons it looks like I m going to be here a while, so it made sense to have my belongings shipped over here. They haven t quite arrived yet, and I ll do another post about that process once they have, but I ve been doing various tweets prefixed with [shipping] during the process. Various people I ve spoken to (some who should know me better) thought this was happening manually. It wasn t. If you care about how it was done, read on. I d been given details of the ship carrying my container, and searching for that turned up the excellent MarineTraffic which let me see the current location of the ship. Turns out ships broadcast their location using AIS and anyone with a receiver can see the info. Very cool, and I spent some time having a look at various bits of shipping around the UK out of interest. I also found the ship s itinerary which give me some idea of where it would be calling and when. Step one was to start recording this data; it was time sensitive and I wanted to be able to see historical data. I took the easy route and set up a cron job to poll the location and itinerary on an hourly basis, and store the results. That meant I had the data over time, if my parsing turned out to miss something I could easily correct it, and that I wasn t hammering Marine Traffic while writing the parsing code. Next I wanted to parse the results, store them in a more manageable format than the HTML, and alert me when the ship docked somewhere or set off again. I ve been trying to learn more Python rather than doing my default action of turning to Perl for these things, and this seemed like a simple enough project to try out. Beautiful Soup seemed to turn up top for HTML parsing in Python, so that formed the basis. Throwing the info into a database so I could do queries felt like the right move so I used SQLite - if this had been more than a one off I d have considered looking at PostgreSQL and its GIS support. Finally Tweepy made it very easy to tweet from Python in about 4 lines of code. The whole thing weighed in at only 175 lines of code, mostly around pulling the info out of the HTML and then a little to deal with checking for state changes against the current status and the last piece of info in the database. The pieces of information I chose to store were the time of the update (i.e. when the ship sent it, not when my script ran), reported area, reported state, the position + course, reported origin, reported destination and eta. The fact this is all in a database makes it very easy to do a few queries on the data. How fast did the ship go? 
sqlite> SELECT MAX(speed) FROM status;
MAX(speed)
21.9
What areas did it report? 
sqlite> SELECT area FROM status GROUP BY area;
area
-
Atlantic North
California
Caribbean Sea
Celtic Sea
English Channel
Hudson River
Pacific North
Panama Canal
What statuses did we see? 
sqlite> SELECT status FROM status GROUP BY status;
status
At Anchor
Moored
Stopped
Underway
Underway using Engine
Finally having hourly data lets me draw a map of where the ship went. The data isn t complete, because the free AIS info depends on the ship being close enough to a receiving station. That means there were a few days in the North Atlantic without updates, for example. However there s enough to give a good idea of just how well traveled my belongings are, and it gave me an excuse to play with OpenLayers.
(Apologies if the zoom buttons aren t working for you here; I think I need to kick the CSS in some manner I haven t quite figured out yet.)

Erich Schubert: Big data predictions for 2015

My big data predictions for 2015:
  1. Big data will continue to fail to deliver for most companies.
    This has several reasons, including in particular: 1: lack of data to analyze that actually benefits from big data tools and approaches (and which is not better analyzed with traditional tools). 2: lack of talent, and failure to attract analytics talent. 3: stuck in old IT, and too inflexible to allow using modern tools (if you want to use big data, you will need a flexible "in-house development" type of IT that can install tools, try them, abandon them, without going up and down the management chains) 4: too much marketing. As long as big data is being run by the marketing department, not by developers, it will fail.
  2. Project consolidation: we have seen hundreds of big data software projects the last years. Plenty of them on Apache, too. But the current state is a mess, there is massive redundancy, and lots and lots of projects are more-or-less abandoned. Cloudera ML, for example, is dead: superseded by Oryx and Oryx 2. More projects will be abandoned, because we have way too many (including much too many NoSQL databases, that fail to outperform SQL solutions like PostgreSQL). As is, we have dozens of competing NoSQL databases, dozens of competing ML tools, dozens of everything.
  3. Hype: the hype will continue, but eventually (when there is too much negative press on the term "big data" due to failed projects and inflated expectations) move on to other terms. The same is also happening to "data science", so I guess the next will be "big analytics", "big intelligence" or something like that.
  4. Less openness: we have seen lots of open-source projects. However, many decided to go with Apache-style licensing - always ready to close down their sharing, and no longer share their development. In 2015, we'll see this happen more often, as companies try to make money off their reputation. At some point, copyleft licenses like GPL may return to popularity due to this.

12 January 2015

Russell Coker: Systemd Notes

A few months ago I gave a lecture about systemd for the Linux Users of Victoria. Here are some of my notes reformatted as a blog post: Scripts in /etc/init.d can still be used, they work the same way as they do under sysvinit for the user. You type the same commands to start and stop daemons. To get a result similar to changing runlevel use the systemctl isolate command. Runlevels were never really supported in Debian (unlike Red Hat where they were used for starting and stopping the X server) so for Debian users there s no change here. The command systemctl with no params shows a list of loaded services and highlights failed units. The command journalctl -u UNIT-PATTERN shows journal entries for the unit(s) in question. The pattern uses wildcards not regexs. The systemd journal includes the stdout and stderr of all daemons. This solves the problem of daemons that don t log all errors to syslog and leave the sysadmin wondering why they don t work. The command systemctl status UNIT gives the status and last log entries for the unit in question. A program can use ioctl(fd, TIOCSTI, ) to push characters into a tty buffer. If the sysadmin runs an untrusted program with the same controlling tty then it can cause the sysadmin shell to run hostile commands. The system call setsid() to create a new terminal session is one solution but managing which daemons can be started with it is difficult. The way that systemd manages start/stop of all daemons solves this. I am glad to be rid of the run_init program we used to use on SE Linux systems to deal with this. Systemd has a mechanism to ask for passwords for SSL keys and encrypted filesystems etc. There have been problems with that in the past but I think they are all fixed now. While there is some difficulty during development the end result of having one consistent way of managing this will be better than having multiple daemons doing it in different ways. The commands systemctl enable and systemctl disable enable/disable daemon start at boot which is easier than the SysVinit alternative of update-rc.d in Debian. Systemd has built in seat management, which is not more complex than consolekit which it replaces. Consolekit was installed automatically without controversy so I don t think there should be controversy about systemd replacing consolekit. Systemd improves performance by parallel start and autofs style fsck. The command systemd-cgtop shows resource use for cgroups it creates. The command systemd-analyze blame shows what delayed the boot process and
systemd-analyze critical-chain shows the critical path in boot delays. Sysremd also has security features such as service private /tmp and restricting service access to directory trees. Conclusion For basic use things just work, you don t need to learn anything new to use systemd. It provides significant benefits for boot speed and potentially security. It doesn t seem more complex than other alternative solutions to the same problems. https://wiki.debian.org/systemd http://freedesktop.org/wiki/Software/systemd/Optimizations/ http://0pointer.de/blog/projects/security.html
Related posts:
  1. systemd a Replacement for init etc The systemd projecct is an interesting concept for replacing init...
  2. Some Notes on DRBD DRBD is a system for replicating a block device across...
  3. licence for lecture notes While attending LCA it occurred to me that the lecture...

10 January 2015

Mike Gabriel: Shifting my Focus in X2Go

Dear X2Go Community, dear friends, as many of you may know, I have been contributing a considerable amount
of time to upstream-maintaining X2Go over the past 4 years. I provided
new X2Go components (Python X2Go, PyHoca X2Go Client, a publicly
available X2Go Session Broker, X2Go MATE Bindings, etc.) and focused on
making X2Go a wide-spread community project. For the last 2-3 years I
have been in the role of the X2Go project coordinator and various other
roles. With the beginning of 2015, I will pass on several of those roles to
other people in the project, see the below list for already assigned and
unassigned roles: The reasons for tremendously reducing my workload on X2Go are these: In several internal exchanges we (Heinz, Stefan, Mihai, Mike#2, read more

9 January 2015

Chris Lamb: Uploading a large number of files to Amazon S3

I recently had to upload a large number (~1 million) of files to Amazon S3. My first attempts revolved around s3cmd (and subsequently s4cmd) but both projects seem to based around analysing all the files first rather than blindly uploading them. This not only requires a large amount of memory, non-trivial experimentation, fiddling and patching is also needed to avoid unnecessary stat(2) calls. I even tried a simple find xargs -P 5 s3cmd put [..] but I just didn't trust the error handling correctly. I finally alighted on s3-parallel-put, which worked out well. Here's a brief rundown on how to use it:
  1. First, change to your source directory. This is to ensure that the filenames created in your S3 bucket are not prefixed with the directory structure of your local filesystem whilst s3-parallel-put has a --prefix option, it is ignored if you pass a fully-qualified source, ie. one starting with a /.
  2. Run with --dry-run --limit=1 and check that the resulting filenames will be correct after all:
$ export AWS_ACCESS_KEY_ID=FIXME
$ export AWS_SECRET_ACCESS_KEY=FIXME
$ /path/to/bin/s3-parallel-put \
    --bucket=my-bucket \
    --host=s3.amazonaws.com \
    --put=stupid \
    --insecure \
    --dry-run --limit=1 \
    .

[..]
INFO:s3-parallel-put[putter-21714]:./yadt/profile.Profile/image/circle/807.jpeg -> yadt/profile.Profile/image/circle/807.jpeg
[..]
  1. Remove --dry-run --limit=1, and let it roll.

8 January 2015

Wouter Verhelst: ExtreMon example

About a month ago, I blogged about extremon. As a reminder, ExtreMon is a monitoring tool that allows you to view things as they are happening, rather than with the ~5 minute delay that munin gives you, and also avoiding the quad-state limitation of Nagios' "good", "bad", "ugly", and "unknown" states. No, they're not really called that. Yes, I know you knew that. Anyway. In my blog post, I explained how you can set up ExtreMon, and I also set up a fairly limited demo version on my own server. But I have since realized that while it is functional, it doesn't actually show why ExtreMon is so great. In an effort to remedy that, I present you an example of what ExtreMon can do. Let's start with a screenshot of the ExtreMon console at the customer for which I spent time trying to figure out how to get it up and running: Click for full sized version. You'll note that even in that full-sized version, many things are unreadable. This is because the ExtreMon console allows one to move around (right mouse button drag for zoom; left mouse button drag for moving around; control+RMB for rotate; center mouse button to reset to default); so what matters is that everything fits on the screen, not whether it is all readable (if you need to read, you zoom). The image shows 18 rectangles. Each rectangle represents a single machine in this particular customer's HPC cluster. The top three rectangles are the cluster's file servers; the rest are its high performance nodes. You'll note that the left fileserver has 8 processor cores (top row), 8 network cards (bottom row, left part), and it also shows information on its memory usage (bottom row, small rectangle in the middle) as well as its NFS client and server procedure calls (bottom row, slightly larger rectangles to the right). This file server is the one on which I installed ZFS a while back; hence the large amount of disks visible in the middle row. The leftmost disk is the root filesystem (which is an ext4 off a hardware RAID1); the two rightmost "disks" are the PCIe-attached SSDs which are used for the ZFS L2ARC and write log. The other disks in this file server nicely show how ZFS does write load balancing over all its disks. The second file server has a hardware RAID1 on which it stores all its data; as such, there is only one disk graph there. It is also somewhat more limited in network, as it has only two NICs. It does, however, also have 8 cores. The last file server has no more than four processor cores; in addition, it also does not have a hardware RAID controller, so it must use software RAID over its four hard disks. This server is used for archival purposes, mostly, since it is insufficient for most anything else. As said, the other nodes are the "compute nodes", where the hard work is done. Most of these compute nodes have 16 cores each; two have 12 instead. When this particular screenshot was taken, four of the nodes (the ones showing red in their processor graphs) were hard at work; the others seem to have been mostly idling. In addition to the familiar memory, NFS (client only), network, and processor graphs, these nodes also show a "swap space" graph (just below the memory one), which seems fine for most nodes, except for the bottom left one (which shows a few bars that are coloured yellow rather than green). The green/yellow/red stuff is supposed to represent the "ok", "warning", "bad" states that would be familiar from Nagios. In this particular case, however, where "processor is busy all the time" is actually a wanted state, a low amount of idleness on the part of the processor isn't actually a problem, on the contrary. I did consider, therefore, to modify the ExtreMon configuration so that the processor graphs would not show red when the system was under high load; however, I found that differences in colour like this actually makes it more easy to see, at a glance, which machines are busy -- and that's one of the main reasons why we wanted to set this up. If you look carefully, you can find a particular processor core in the graph which shows 100% usage for "idle", "system", and "softirq", at the same time. Obviously that can't be the case, so there's a bug somewhere. Frank seems to believe it is a bug in CollectD; I haven't verified that. At any rate, though, this isn't usually a problem, due to the high update frequency of ExtreMon. The amount of data that's flowing through ExtreMon is amazing: Which renders a grand total of 2887 data points that are shown in this particular screenshot; and then I'm not even counting all the intermediate values, some of which also pass through ExtreMon. Nor am I counting the extra bits which have since been added (this screenshot is a few days old, now, and I'm still finetuning things). Yet even so, ExtreMon manages to update those values once every few seconds, in the worst case. As a result, the display isn't static for a moment, constantly moving and updating data so that what you see is never out of date for more than a second or two. Awesome.

Stefano Zacchiroli: #JeSuisCharlie - RIP Bernard Maris

R.I.P. Bernard Maris and his thoughts on research and the sharing economy via Le Monde, 16 Sep 2014: Le Monde: Que devrait tre une politique de gauche? Une r gulation du capitalisme ou une politique de rupture radicale avec ce syst me conomique? B.M.: [ ] Nous allons vers une conomie de partage, de la gratuit , du logiciel libre en effet. La figure centrale de demain sera le chercheur qui, lorsqu'il donne quelque chose la communaut , ne le perd pas. Le chercheur r pond aux besoins fondamentaux de l'homme: la cr ation, la curiosit , le changement, le progr s. Il est oblig de coop rer. La coop ration canalise la violence, que le lib ralisme esp rait canaliser par le doux commerce! L'au-del du capitalisme sera une conomie solidaire et fraternelle. Aujourd'hui, la question incontournable porte sur la nature du travail.[ ]
Bernard Maris
23 Sep 1946 - 7 Jan 2015
#JeSuisCharlie
translate to English (via Google Translate)

Russell Coker: Conference Suggestions

LCA 2015 is next week so it seems like a good time to offer some suggestions for other delegates based on observations of past LCAs. There s nothing LCA specific about the advice, but everything is based on events that happened at past LCAs. Don t Oppose a Lecture Question time at the end of a lecture isn t the time to demonstrate that you oppose everything about the lecture. Discussion time between talks at a mini-conf isn t a time to demonstrate that you oppose the entire mini-conf. If you think a lecture or mini-conf is entirely wrong then you shouldn t attend. The conference organisers decide which lectures and mini-confs are worthy of inclusion and the large number of people who attend the conference are signalling their support for the judgement of the conference organisers. The people who attend the lectures and mini-confs in question want to learn about the topics in question and people who object should be silent. If someone gives a lecture about technology which appears to have a flaw then it might be OK to ask one single question about how that issue is resolved, apart from that the lecture hall is for the lecturer to describe their vision. The worst example of this was between talks at the Haecksen mini-conf last year when an elderly man tried at great length to convince me that everything about feminism is wrong. I m not sure to what degree the Haecksen mini-conf is supposed to be a feminist event, but I think it s quite obviously connected to feminism which is of course was why he wanted to pull that stunt. After he discovered that I was not going to be convinced and that I wasn t at all interested in the discussion he went to the front of the room to make a sexist joke and left. Consider Your Share of Conference Resources I ve previously written about the length of conference questions [1]. Question time after a lecture is a resource that is shared among all delegates. Consider whether you are asking more questions than the other delegates and whether the questions are adding benefit to other people. If not then send email to the speaker or talk to them after their lecture. Note that good questions can add significant value to the experience of most delegates. For example when a lecturer appears to be having difficulty in describing their ideas to the audience then good questions can make a real difference, but it takes significant skill to ask such questions. Dorm Walls Are Thin LCA is one of many conferences that is typically held at a university with dorm rooms offered for delegates. Dorm rooms tend to have thinner walls than hotel rooms so it s good to avoid needless noise at night. If one of your devices is going to make sounds at night please check the volume settings before you start it. At one LCA I was startled at about 2AM but the sound of a very loud porn video from a nearby dorm room, the volume was reduced within a few seconds, but it s difficult to get to sleep quickly after that sort of surprise. If you set an alarm then try to avoid waking other people. If you set an early alarm and then just get up then other people will get back to sleep, but pressing snooze repeatedly for several hours (as has been done in the past) is anti-social. Generally I think that an alarm should be at a low volume unless it is set for less than an hour before the first lecture in which case waking people in other dorm rooms might be doing them a favor. Phones in Lectures Do I need to write about this? Apparently I do because people keep doing it! Phones can be easily turned to vibrate mode, most people who I ve observed taking calls in LCA lectures have managed this but it s worth noting for those who don t. There are very few good reasons for actually taking a call when in a lecture. If the hospital calls to tell you that they have found a matching organ donor then it s a good reason to take the call, but I can t think of any other good example. Many LCA delegates do system administration work and get calls at all times of the day and night when servers have problems. But that isn t an excuse for having a conversation in the middle of the lecture hall while the lecture is in progress (as has been done). If you press the green button on a phone you can then walk out of the lecture hall before talking, it s expected that mobile phone calls sometimes have signal problems at the start of the call so no-one is going to be particularly surprised if it takes 10 seconds before you say hello. As an aside, I think that the requirement for not disturbing other people depends on the number of people who are there to be disturbed. In tutorials there are fewer people and the requirements for avoiding phone calls are less strict. In BoFs the requirements are less strict again. But the above is based on behaviour I ve witnessed in mini-confs and main lectures. Smoking It is the responsibility of people who consume substances to ensure that their actions don t affect others. For smokers that means smoking far enough away from lecture halls that it s possible for other delegates to attend the lecture without breathing in smoke. Don t smoke in the lecture halls or near the doorways. Also using an e-cigarette is still smoking, don t do it in a lecture hall. Photography Unwanted photography can be harassment. I don t think there s a need to ask for permission to photograp people who harass others or break the law. But photographing people who break the social agreement as to what should be done in a lecture probably isn t. At a previous LCA a man wanted to ask so many questions at a keynote lecture that he had a page of written notes (seriously), that was obviously outside the expected range of behaviour but probably didn t justify the many people who photographed him. A Final Note I don t think that LCA is in any way different from other conferences in this regard. Also I don t think that there s much that conference organisers can or should do about such things.
Related posts:
  1. A Linux Conference as a Ritual Sociological Images has an interesting post by Jay Livingston PhD...
  2. Suggestions and Thanks One problem with the blog space is that there is...
  3. Length of Conference Questions After LCA last year I wrote about speaking stacks and...

6 January 2015

Steve McIntyre: Bootstrapping arm64 in Debian

I promised to write about this a long time, ooops... :-) Another ARM port in Debian - yay! arm64 is officially a release architecture for Jessie, aka Debian version 8. That's taken a lot of manual porting and development effort over the last couple of years, and it's also taken a lot of CPU time - there are ~21,000 source packages in Debian Jessie! As is often the case for a brand new architecture like arm64 (or AArch64, to use ARM's own terminology), hardware can be really difficult to get hold of. In time this will cease to be an issue as hardware becomes more commoditised, but in Debian we really struggled to get hold of equipment for a very long time during the early part of the port. First bring-up in Debian Ports To start with, we could use ARM's own AArch64 software models to build the first few packages. This worked, but only very slowly. Then Chen Baozi and the folks running the Tianhe-2 supercomputer project in Guangzhou, China contacted us to offer access to some arm64 hardware, and this is what Wookey used for bootstrapping the new port in the unofficial Debian Ports archive. This has now become the normal way for new architectures to get into Debian. We got most of the archive built in debian-ports this way, and we could then use those results to seed the initial core set of packages in the main Debian archive. Second bring-up - moving into the main Debian archive By the time that first Debian bring-up was done, ARM was starting to produce its own "Juno" development boards, and with the help of my boss^4 James McNiven we managed to acquire a couple of those machines for use as official Debian build machines. The existing machines in China were faster, but for various reasons quite difficult to maintain as official Debian machines. So I set up the Junos as buildds just before going to DebConf in August 2014. They ran very well, and (for dev boards!) were very fast and stable. They built a large chunk of the Debian archive, but as the release freeze for Jessie grew close we weren't quite there. There was a small but persistent backlog of un-built packages that were causing us issues, plus the Juno machines are/were not quite suitable as porter boxes for Debian developers all over the world to use for debugging their packages on the new architecture. More horsepower - Linaro machines This is where Linaro came to our aid. Linaro's goal is to help improve Free and Open Source Software on ARM, and one of the more recent projects in Linaro is a cluster of servers that are made available for software developers to use to get early access to ARMv8 (arm64) hardware. It's a great way for people who are interested in this new architecture to try things out, port their software or indeed just help with the general porting effort. As Debian is seen as such an important part of the FLOSS ecosystem, we managed to negotiate dedicated access to three of the machines in that cluster for Debian's use and we set those up in October, shortly before the freeze for Jessie. Andy Doan spent a lot of his time getting these machines going for us, and then I set up two of them as build machines and one as the porter box we were still needing. With these extra machines available, we quickly caught up with the ever-busy "Needs-Build" queue and we've got sufficient build power now to keep things going for the Jessie release. We were officially added to the list of release architectures at the Cambridge mini-Debconf in November, and all is looking good now! And in the future? I've organised the loan of another arm64 machine from AMD for Debian to use for further porting and/or building. We're also expecting that more and more machines will be coming out soon as vendors move on from prototyping to producing real customer equipment. Once that's happened, more kit will be available and everybody will be able to have arm64-powered computers in the server room, on their desk and even inside their laptop! Mine will be running Debian Jessie... :-) Thanks! There's been a lot of people involved in the Debian arm64 bootstrapping at various stages, so many that I couldn't possibly credit them all! I'll highlight some, though. :-) First of all, Wookey's life has revolved around this port for the last few years, tirelessly porting, fixing and hacking out package builds to get us going. We've had loads of help from other teams in Debian, particularly the massive patience of the DSA folks with getting early machines up and running and the prodding of the ftpmaster, buildd and release teams when we've been grinding our way through ever more package builds and dependency loops. We've also had really good support from toolchain folks in Debian and ARM, fixing bugs as we've found them by stressing new code and new machines. We've had a number of other people helping by filing bugs and posting patches to help us get things built and working. And (last but not least!) thanks to all the folks who've helped us beg and borrow the hardware to make the Debian arm64 port a reality. Rumours of even more ARM ports coming soon are entirely scurrilous... *grin*

Holger Levsen: 20150106-lts-december-2014

My LTS December In December 2014 I spent 11h on Debian LTS work and managed to get six DLAs released and another one almost done... I did: Thanks to everyone who is supporting Squeeze LTS in whatever form! Even just expressing that you or the company or project you're working with is using LTS, is useful, as it's always nice to hear once work is used and appreciated. If you can contribute more, please do so. If you can't, that's also fine. It's free software after all :-)

Russ Allbery: Review: Code Complete, Second Edition

Review: Code Complete, Second Edition, by Steve McConnell
Publisher: Microsoft
Copyright: June 2004
ISBN: 0-7356-1967-0
Format: Kindle
Pages: 960
As mentioned in the title, this is a review of the second edition of Code Complete, published in 2004. There doesn't appear to be a later edition at the time of this writing. I should say, as a prefix to this review, that I'm the sort of person who really likes style guides. When learning a language, a style guide is usually the second or third document I read. I enjoy debates over the clearest way to express a concept in code, trying to keep all the code in a large project consistent, and discussing the subtle trade-offs that sit on the boundary between mechanical style issues and the expressiveness of programming. I try to spend some time reading good code and getting better at expressing myself in code. Presumably, therefore, I'm the target audience for this book. It sounded good from the descriptions, so I picked it up during one of the Microsoft Press sales. The stated goal of Code Complete is to collect in one place as much as possible of the oral tradition and lore of the programming field, to try to document and communicate the techniques and approaches that make someone a good programmer. The table of contents sounds like a style guide, with entire sections on variables and statements in addition to topics like how to improve existing code and how to design a new program. If you're starting to think that a 960 page style guide sounds like a bad idea, you're wiser than I. (In my defense, I grabbed this as an ebook and didn't realize how large it was before I bought it.) I have not actually finished this book. I hate to do this: I don't like reviewing books I haven't finished (this will be the first), and I hate starting books and not finishing them. (This is probably not particularly wise, since some books aren't worth finishing, but I've gotten into a rhythm of reading and reviewing that works for me, so I try not to mess with it.) But I've been trying to finish this book off and on for about a year, I don't think it's worth the time investment, and I think I've gotten far enough in it to provide some warnings by others who are deceived by the very high ratings that it gets on Amazon and other places. The primarily problem with Code Complete is its sheer, mind-numbing comprehensiveness. It tries to provide a set of guidelines and a checklist to think about at each level of writing code. This is one of those ideas that might sound good on paper, but which completely doesn't work for me. There is no way I'm going to keep this many rules in my head, in the form of rules, while programming. Much of good style has to be done by feel, and the book I'm looking for is one that improves my feel and my sense of taste for code. What Code Complete seems to provide instead is a compilation of every thought that McConnell has ever had about programming. There's a lot of basic material, a few thoughtful statements, a ton of style advice, an endless compilation of trade-offs and concepts that one should keep in mind, and just a massive, overwhelming pile of stuff. Each chapter (and there are a lot of chapters) ends in a checklist of things that you should think about when doing a particular programming task. To give you a feel for the overwhelming level of trivia here, this is the checklist at the end of the chapter where I stopped reading, on quality assurance in software. This is one picked at random; a lot of them are longer than this.
  • Have you identified specific quality characteristics that are important to your project?
  • Have you made others aware of the project's quality objectives?
  • Have you differentiated between external and internal quality characteristics?
  • Have you thought about the ways in which some characteristics might compete with or complement others?
  • Does your project call for the use of several different error-detection techniques suited to finding several different kinds of errors?
  • Does your project include a plan to take steps to assure software quality during each stage of software development?
  • Is the quality measured in some way so that you can tell whether it's improving or degrading?
  • Does management understand that quality assurance incurs additional costs up front in order to save costs later?
I'm not saying those are bad things to think about with quality assurance, but you may notice a few issues immediately. They're very general and vague, they're not phrased in a particularly compelling or memorable way, and there are a lot of them. This falls between two stools: it's too much for the programmer who is thinking about quality as part of an overall project but not focusing on it (particularly when you consider that the book is full of checklists like this for everything from variable naming to how to structuring if statements to program debugging), but it's not nearly specific or actionable enough for someone who is focusing on quality assurance. It's not that the information isn't organized: there's a lot of structure here. And there are bits and pieces here that are occasionally interesting. McConnell is very data-driven and tries to back up recommendations with research on error rates and similar concrete measurements. It's just insufficiently filtered and without elegant or memorable summary. There is far too much here, an overwhelming quantity, and hopelessly mixed between useful tidbits and obvious observations that anyone who has been programming for a while would pick up, all presented in the same earnest but dry tone. It didn't help that there's a lot here I didn't agree with. Some of that is to be expected: I've never agreed completely with any style guide. But McConnell kept advocating variable and function naming conventions that I find rather ugly and tedious, and the general style of code he advocates feels very "bureaucratic" to me. It's not exactly wrong, but one of the things that I look for in style discussions is to be inspired by the elegant and simple way someone finds to phrase something in code. A lot of the code in this book just felt mind-numbing. It's functional, but uninteresting; perfectly adequate for a large project, but not the sort of discussion that inspires me to improve the quality of my craft. So, I didn't finish this. I gave up about halfway through. It's frustrating, since I was occasionally finding an interesting nugget of information. But they were too few and far between, and the rest of the book was mostly... boring. It's possible that I just know too much about programming to be the person for whom that McConnell was writing this book. It's certainly true that the book has not aged particularly well; it's focused on fairly old-school languages (C, C++, Java, and Visual Basic) and says almost nothing about modern language techniques, although it does have a bit about extreme programming. But whatever the reason is, it didn't work for me at all. I would rate it as one of the worst books about programming I've tried to read. And that's notably different enough from its reviews that it seems worth throwing this out there as a warning. I'm quite disappointed, since I'd heard nothing but praise for this book before picking it up. But it's not for me, and I'm now dubious of its value for any programmer outside of a fairly narrow, large-team, waterfall development process involving large numbers of people writing very large quantities of code in languages that aren't very expressive. And, well, in that situation I think one would get more benefit from changing that environment than reading this book. Rating: unfinished

5 January 2015

Russ Allbery: Review: Ancillary Sword

Review: Ancillary Sword, by Ann Leckie
Series: Imperial Radch #2
Publisher: Orbit
Copyright: October 2014
ISBN: 0-316-24665-4
Format: Trade paperback
Pages: 354
This is the second book in the Imperial Radch series and a direct sequel to Ancillary Justice. You don't want to read this book out of order, since the previous book sets up the background of everything that happens here. Besides, Ancillary Justice is an amazing book. It's going to be challenging to review Ancillary Sword without spoiling the previous book. If you're planning on reading Ancillary Justice but haven't gotten to it, you may want to stop here and come back to this review after you've read it. Or, even better, just read both books. They're some of the best science fiction I've read. Ancillary Justice started small, with one person and their quixotic search for revenge, and grew large, to encompass conflicts and confrontations that would shake the Radch. Ancillary Sword returns to a smaller scale and stays there. This means that much of what was left unresolved at the end of the previous book is still unresolved; Leckie does not continue escalating into large-scale conflict. It also means that we see a lot more of Breq making personal choices and trying to work out her own sense of morality, plus semi-adopting a couple more injured people along the way. One of my favorite types of stories is where I get to watch someone who is very good at something do the thing that they're very good at. Breq's unique background and experience makes her a wildcard outsider with vast experience in her new role. (Not to mention the special advantages she has from her implants.) Her long experience with people, similarly from a unique perspective, lets her use her power to effectively navigate political situations while keeping people slightly off-balance. And now she has some real power, made more potent for being somewhat ill-defined. In short, this is a story of political agency, given to someone who hasn't had it before but who is very good at using it. It's immensely satisfying, in part because it's not a simple wish fulfillment. Breq can't just reshape the world to her preferences; in fact, she can't do much about one of the social conflicts she runs into, except treat the people involved with unexpected respect. But she can occasionally do something, and she can always upset existing power structures in subtle ways, and the way Leckie writes this makes it so much fun to read. I think one of the reasons why I enjoyed this so much is that Breq is not relentlessly introspective. She just acts. Usually this sort of book involves lots of soul-searching and analysis, and the lack is refreshing. The other people in the story analyze Breq much more than she analyzes herself, sometimes incorrectly, and Breq finds the whole thing faintly amusing. Not only does this keep the story from bogging down in too much internal drama, it means that Breq frequently surprises the reader, usually in ways that had me grinning. And, despite not mulling things over incessantly, she is growing and developing, finding her own sense of morality and and ethics in a way that's sometimes only apparent in retrospect. The one caveat I will mention is that this is a book that concerns itself a great deal with colonialism and racial slavery, but it's a fantasy of political agency focused on someone who's part of the dominant culture. While it's not quite accurate to say that Breq is this world's equivalent of white, she can pass, and she's Radchaai. I thought the book handles the issues reasonably well, but it is still using oppressed cultures to focus on the agency and power of someone who is, comparatively, privileged. This didn't bother me while I was reading the story, but it started to bother me a little afterwards once it was pointed out. There's nothing inherently wrong with that story, but it's a rather common pattern, and I'm afraid Ancillary Sword doesn't do much to broaden the pattern. That said, it's a caveat rather than a fatal flaw, at least for me. Ancillary Sword is obviously the middle book of a trilogy, and normally the lack of forward progress on the overarching story and the sense of filling in background and setting the scene would undermine the book. But Breq and the other characters in this world are so fascinating that I didn't mind. The ending was not quite what I expected, but worked better the more that I thought about it. I'm really looking forward to the next book. Followed by Ancillary Mercy. Rating: 9 out of 10

4 January 2015

Russ Allbery: Review: Programming Ruby

Review: Programming Ruby, by Dave Thomas, et al.
Publisher: Pragmatic Bookshelf
Copyright: 2005
Printing: May 2006
ISBN: 0-9745140-5-5
Format: Trade paperback
Pages: 785
There are a few different editions of this book. The version I read is the second edition, written by Dave Thomas with Chad Fowler and Andy Hunt and published in 2005, covering Ruby 1.8.2. There's now a fourth edition, covering Ruby 1.9 and 2.0, which is probably what you'd want if you were buying this book today. This book, in whatever edition, is called the Pickaxe in the Ruby community after its cover. I've used a lot of different programming languages, so I can usually pick one up on the fly reasonably well, but I still like to read a good introductory book before using one seriously. It's a bit too easy to get lost or to fall into habits that don't match the best practices of the language community without a solid introduction. I've been using a bit of Ruby off and on since I started using Puppet, but I'm looking at doing more serious development using Chef, so I decided it was time to get that introduction. (It helped that I had this book sitting around, although that's also why I read an older edition.) Programming Ruby starts with the obligatory introduction to installing and running Ruby, and then provides a high-level introduction to the language and its basic types just enough to make Ruby comprehensible before starting into the object system. Everything is an object in Ruby, so the book introduces the object system as early as possible, and then shows the rest of the language from constants up in the light of that object system. The rest of part one follows the normal language introduction path, building up from constants and methods to exceptions, modules, and basic IO. It closes with chapters about threads and processes, unit testing, and the debugger. Part two is a grab-bag of one-chapter topics describing how to use Ruby in a particular setting, or showing one angle of the language. The best of those chapters for me was the one on RDoc, partly because I'm quite impressed by Ruby's documentation system. A few of these chapters are oddly in-depth for an introductory book I doubt I'm ever going to use all the details about special irb configuration, and if I do, I'd just look them up but I greatly appreciated the solid chapter on how to write Ruby extensions in C. There is also the obligatory chapter on writing GUI applications with Tk, which always seems to show up in these sorts of introductions and which always baffles me. Does anyone actually do this any more instead of writing a web application? Part three dives back into the language and provides a more complete and formal description. The authors aren't afraid to get into some of the internals, which I appreciated. There is a good chapter here on the details of the type system and how objects and classes interact, and a much-needed extended discussion of duck typing. This type of weak typing and runtime binding is fundamental to how Ruby approaches objects, for better or worse. (I have mixed opinions; it makes some things easier, but I increasingly appreciate strong typing and more formal interface definitions.) Some discussion of marshalling and introspection closes out the discussion portion of the book. That's about 420 pages of the material. The rest of the book is a detailed reference on all of the core classes, and a quicker overview of the standard library. Normally, this sort of thing is thrown into language introductions to pad out the page count, but usually the language's official documentation is better at this sort of reference. But I found Programming Ruby to be an exception. The reference is succinct, sticking to a paragraph or two for each method, and did a great job of providing enough cross-reference and discussion to put each class into a broader perspective. It's the most useful example of this type of reference section I've seen. I still probably won't use it after this initial reading, but I think I got a better feel for the language from reading through it. It's hard to review a book like this without reviewing the language it documents, at least a little bit. I'll indulge: it entertains me how much Ruby is obviously based on Perl, including borrowing some of Perl's more dubious ideas. The global punctuation variables will look familiar to any Perl programmer, and the oddly-named global variables for the interpreter flags are in the same spirit. The language unfortunately has similar problems as Perl with safely running commands without using the shell; it's possible, but not the default and not what the built-ins do. There are places where I wish Ruby were a little less like Perl. The plus side for an experienced Perl programmer is that Ruby feels quite familiar and has made some clear improvements. The ? and ! convention for methods that return booleans or modify objects in-place is brilliant in its simplicity, and something I'd love to see in more languages. And the way Ruby implements ubiquitous code blocks for both iterators and for any temporary objects is lovely once one gets used to it. It's similar to Python's context managers, except more general and built deeper into the language. Returning to the review of the book, rather than the topic, Programming Ruby has a good, clear explanation of blocks, iterators, and yield. If you're interested in getting a grounding in Ruby, this book still feels like a solid introduction. The edition I read is getting a bit long in the tooth now that we're on Ruby 2.1, but the pace of language change has slowed, and most of the book is still applicable. (If you're buying it new, you should, of course, get the later edition.) The table of contents makes it seem like the book is covering the same ground multiple times, but that organizational strategy worked better than I expected. Ruby is not the most organized language in the world, so I still felt a bit overwhelmed with random method names in places, but I never felt lost in the mechanics of the language. In short, recommended if you want a good introduction to the language, although probably in a later edition. Rating: 8 out of 10

2 January 2015

Gunnar Wolf: Status of the OpenPGP keyring: 1024D is a thing of the past!

Having seen the end of December and the beginning of January, this is the time of year where we say "Happy new year!" But this is a very interesting new year: We have also went past our much announced deadline for the <2048 bit keys to be removed from the Debian keyrings. And yes, our highly efficient keyring-maint team managed to deliver on the promised time And, I'd say, with much success. Lets see the numbers Only before that, refer to Jonathan's mail to debian-devel-announce for further, fuller information. So, first of all, how do overall numbers look? Just remember, the following are not the number of DDs, just the number of active keys. That is, the holders to the 252 DD and 35 DM keys we removed are still valid Debian Developers/Maintainers, but have to get a new key accepted to perform many of their tasks in the project. The graph above shows the sharp change between tags 2014.12.31 and 2015.01.01. But my definition of success is that we managed to get the number down to just 252+35=287 from what we had back in August, when we did our DebConf presentation and started the aggressive push: 490 DD keys and 49 DM keys. Since then, 34 DDs requested their retirement, becoming emeritus, and practically all of the rest managed to get their key transition done! So, lets go again easiest-to-hardest. First, the Non-uploading Debian Developers keyring: As this is the newest keyring in existence, and is also the smallest one, we were already without <2048 keys since 2011. Nothing to see, move along. Then, as for the Debian Maintainers: We did have a sensible migration from weaker to stronger keys, but it was not as sharp as I'd have liked. That makes sense, after all, since DMs have less involvement and compromise in the project in regard to DDs. So, we only processed 15 DM keys since August, which is almost a third of the keys we needed to process to reach the ideal 100% migration. Now, as for our biggest and oldest keyring, and the one that denotes more project involvement, here is the graph for the uploading Debian Developers: And yes, here you can see the sharp turn we saw in the second half of this year: By DebConf time, we were happy because the red and yellow lines had just crossed. But we were still sitting at 490 DD keys needing to be migrated. Half of the DD keys (compared to almost a fourth for the DM keys). I'm almost sure we anticipated in our presentation (I know, I should check the video) that, by January 1st, we would have to retire around 300 keys. And I'm very, very happy and proud that we managed to get the number down to 252. And, yes, people leave things to the end: We already have some more pending requests in the Request Tracker to introduce new keys for our fellow friends who were disabled. We will be working to make keyring pushes more frequent than our usual monthly uploads until requests go back to a sane level. So, if everything runs smoothly, this will probably be the last of my posts in this regard. This has been quite an interesting (and exhausting!) experience!

Russ Allbery: Review: An Imaginary Tale

Review: An Imaginary Tale, by Paul J. Nahin
Publisher: Princeton University
Copyright: 1998, 2007
Printing: 2010
ISBN: 0-691-14600-4
Format: Trade paperback
Pages: 259
Subtitled The Story of -1, An Imaginary Tale is a history of the use of the number mathematicians normally call i. Nahin starts in the 15th century with the discussion of the time on solving cubic equations, specifically del Ferro's solution to the depressed cubic. He walks through how to approach that solution with imaginary numbers, provides a brief introduction to the complex number plane, and then explains that del Ferro didn't follow the logic in those directions at all. Mathematicians at the time were dubious about negative numbers because they were not intuitive representations of real-world quantities. The square root of negative numbers was considered simply impossible. Nahin continues on in this historical vein for three chapters, walking through the mathematical problems that arose from analysis of cubics and the constant appearance of imaginary numbers, the early attempts to find a geometric interpretation, and then the slow development of the modern conception of imaginary numbers in the 19th century. The emphasis throughout is on the specifics of the math, not on the personalities, although there are a few personal tidbits. Along the way, he takes frequent side journeys to highlight the various places complex numbers are useful in solving otherwise-intractable problems. After that initial history come two chapters of applications of complex numbers: vector analysis, Kepler's laws, applications in electrical engineering, and more. He does win a special place in my heart by walking through the vector analysis problem that George Gamow uses to demonstrate complex numbers in One Two Three... Infinity: a treasure map whose directions depend on landmarks that no longer exist. Following that is a great chapter on deeper mathematical wizardry involving i, including Euler's identity, ii, and a pretty good explanation of hyperbolic functions. The final chapter is an introduction to complex function theory. One's opinion of this book is going to vary a lot depending on what type of history of math you like to read. Unfortunately, it wasn't what I was hoping for. That doesn't make it a bad book other reviewers have recommended it highly, and I think it would be a great book for someone with slightly different interests. But it's a very mathematical book. It's full of proofs, calculations, and analysis, and assumes that you remember a reasonable amount of algebra and calculus to follow along. It's been a long time since I studied math, and while I probably could have traced the links between steps of his proofs and derivations with some effort, I found myself skimming through large chunks of this book. I love histories of mathematics, and even popularizations of bits of it (particularly number theory), but with the emphasis on the popularization. If you're like me and are expecting something more like The Music of the Primes, or even One Two Three... Infinity, be warned that this is less about the people or the concepts and more about the math itself. If you've read books like that and thought they needed more equations, more detail, and more of the actual calculations, this may be exactly what you're looking for. Nahin is fun to read, at least when I wasn't getting lost in steps that are obvious to him. He's quite enthusiastic about the topic and clearly loves being able to show how to take apart a difficult mathematical equation using a novel technique. His joy reminds me of when I was most enjoying my techniques of integration class in college. Even when I started skimming past the details, I liked his excitement. This wasn't what I was looking for, so I can't exactly recommend it, but hopefully this review will help you guess whether you would like it. It's much heavier on the mathematics and lighter on the popularization, and if that's the direction you would have preferred the other similar books I've reviewed to go, this may be worth your attention. Rating: 6 out of 10

John Goerzen: Sound players: Adventures with Ampache, mpd, pulseaudio, Raspberry Pi, and Logitech Media Server

I finally decided it was about time to get my whole-house sound project off the ground. As an added bonus, I d like to be able to stream music from my house to my Android phone. Some Background It was about 2.5 years ago that I last revisited the music-listening picture on Linux. I used Spotify for awhile, but the buggy nature of its support for local music eventually drove me up the wall. I have a large collection of music that will never be on Spotify (local choirs, for instance) and this was an important feature. When Google Play Music added the feature of uploading your local collection, I used that; it let me stream music from my phone in my car (using the Bluetooh link to the car). I could also listen at home on a PC, or plug my phone into various devices to play. But that was a hassle, and didn t let me have music throughout the house. Google Play is reasonable for that, but it has a number of really glaring issues. One is that it often gets album artwork wrong; it doesn t use the ID3 tags embedded in the files, but rather tries to guess . Another is that the sync is only add . Move files to another place in your collection, or re-rip them to FLAC and replace your old MP3s, and suddenly they re in Google Play twice. It won t ever see updated metadata, either quite a hassle for someone that uses Musicbrainz and carefully curates metadata. My Hardware I already have an oldish PC set up as a entertainment box running MythTV. It is a diskless system (boots PAE over the network and has NFS root) and very quiet. It has video output to my TV, and audio output via S/PDIF to my receiver. It is one logical audio frontend. My workstation in my office is another obvious place. My kitchen has a radio with a line in jack, and I also have a small portable speaker with a line in jack to make the last two options. I also have a Raspberry Pi model B that I bought awhile ago and was looking for a purpose, so I thought this should be cheap and easy, right? Well. Cheap, yes. Easy, not so much. First attempt: Ampache The Ampache project produces quite a nice piece of software. Ampache has matured significantly in the last 2.5 years, and the usability of its web-interface with HTML5 and Flash players as options is quite impressive. It is easily as usable as Google s, though its learning curve is rather more steep. There are multiple Android apps for Ampache to stream remotely. And, while most are terribly buggy and broken, there is at least one that seems to work well. Ampache can also output m3u/pls files for a standalone player. It does on-the-fly transcoding. There are some chinks in the armor, however. The set of codecs that are transcoded or passed through is a global setting, not a per-device setting. The bitrates are per-account, so you can t easily have it transcode FLAC into 320Kbps mp3 for streaming on your LAN and 128Kbps MP3 for streaming to your phone. (There are some hacks involving IP address ranges and multiple accounts, but they are poorly documented and cumbersome.) Ampache also has a feature called localplay in which it drives local players instead of remote ones. I tried to use this in combination with mpd to drive music to the whole house. Ampache s mpd interface is a bit odd; it actually loads things up into mpd s queue. Sadly it shares the same global configs as the rest. Even though mpd is perfectly capable of handling FLAC audio, the Ampache web player isn t, so you have to either make it transcode mp3 for everything or forego the web player (or use a second account that has transcode everything set). Frustratingly, not one of the Android clients for Ampache is even remotely compatible with Localplay, and some will fail in surprising ways if you have been using Localplay on the web client. So let s see how this mpd thing worked out. Ampache with mpd for whole-house audio The primary method here is to use mpd s pulseaudio driver. I configured it like so:
audio_output
type "pulse"
name "MPD stream"
#server "remote_server" # optional
sink "rtp" # optional
mixer_type "software"

 Then in /etc/pulse/system.pa:
load-module module-null-sink sink_name=rtp format=s16be channels=2 rate=44100 sink_properties="device.description='RTP Multicast Sink'"
load-module module-rtp-send source=rtp.monitor
 This tells Pulse to use multicast streaming to the LAN for the audio packets. Pulse is supposed to have latency synchronization to achieve perfect audio everywhere. In practice, this works somewhat poorly. Plus I have to install pulse everywhere, which inserts its tentacles way too deeply into the ALSA stack for my taste. (alsamixer suddenly turns useless by default, for instance.) But I gave it a try. After much fiddling Pulse is rather poorly documented and the interactions of configuration tools with it even more so I found a working configuration. On my MythTV box, I added:
load-module module-rtp-recv
load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1;x.x.x.x
 The real annoyance was getting it to set the output through the S/PDIF digital port. Finally I figured out the magic potion:
set-card-profile 0 output:iec958-stereo+input:analog-stereo
 This worked reasonably well. The Raspberry Pi was a much bigger challenge, however. I put Raspbian on it easily enough, and installed Pulse. But apparently it is well-known in the Pulse community that Pulse s RTP does not work well with wifi. Multicast itself works poorly with wifi in general, and Pulse won t do unicast RTP. However, Pulse assumes very low latency and just won t work well with wifi at all. Morever, pulseaudio on the rpi is something of a difficult beast to tame. It has crackling audio, etc. I eventually got it working decently with: daemon.conf:
realtime-scheduling = yes
realtime-priority = 5
resample-method = src-sinc-fastest
default-sample-rate = 44100
default-fragments = 4
default-fragment-size-msec=20
 and in system.pa, commented out the module-udev-detect and module-detect, and added:
load-module module-alsa-card device_id=0 tsched=0 fragments=10 fragment_size=640 tsched_buffer_size=4194384 tsched_buffer_watermark=262144
 plus the rtp-recv and tcp protocol lines as before. This got it working with decent quality, but it was always out of sync by at least a few tenths of a second, if not a whole second, with the other rooms. Brief diversion: mplayer Many players besides pulseaudio can play the RTP streams that pulseaudio generates. mplayer, for instance, can. I found this worked on the Raspberry Pi:
mplayer -really-quiet -cache 64 -cache-min 95 -demuxer rawaudio -rawaudio format=0x20776172 rtp://224.0.0.56:46510
 But this produced even worse sync. It became clear that Ampache was not going to be the right solution. Logitech Media Server I have looked at this a few times over the years, but I ve somehow skipped it. But I looked at it again now. It is an open source (GPL) server, and was originally designed to work with Logitech hardware. There are now all sorts of software clients out there. There is a helpful wiki about it, although Logitech has rebranded the thing so many times, it s not even consistent internally on what the heck it s called (is it LMS? or Squeezebox Server? or Slimserver? The answer is: yes.) LMS does no web-based streaming. At all. But I thought I d give it a try. Installation is a little weird (its .deb packages up binary perl modules for things that are already in Debian, for half a dozen architectures and many Perl versions.) I had some odd issues but eventually it worked. It scanned my media collection. I installed squeezelite on my workstation as my first player, and things worked reasonably well out of the box. I proceeded to install squeezelite on the Raspberry Pi and MythTV server (where I had to compile libsoxr for wheezy, due to an obscure error I couldn t find the cause of until I used strace). Log in to the LMS web interface, tell it to enable synchronization, and bam! Perfect synchronization! Incredible. But I ran into two issues: one was that it used a lot of CPU even on my workstation (50% or more, even when idle). Strangely, it used far less CPU on the pi than did pulseaudio. Secondly, I d get annoying clicks from it from time to time. Some debugging and investigation revealed that they were both somewhat related; it was getting out of sync with the pi and correcting. Initially I assumed this to be an issue with the pi, but tracked it down to something else. I told it to upsample to 48kHz and this made the problem go away. My command on PC hardware is:
squeezelite -a 200:10 -o dmix -u hLX -z
 And on the Raspberry Pi:
squeezelite -a 800:10:16:0 -n kitchen -z
 It is working perfectly now. LMS even has an Android remote control app that can control all devices together, queue up playlists, etc. Very slick. The Raspberry Pi The Raspberry Pi, even the model B+, apparently has notoriously bad audio output. I noticed this first as very low volume on the output. I found a $11 USB DAC on Amazon that seems to do the trick, so we will see. I am also using the $10 Edimax USB Wifi and it seems to work well. Other Options The big-name other option here is Sonos. They have perfect sync audio already working. But they re pricy, closed, and proprietary; you have to buy Sonos speakers, transmitters, etc. for the entire house. Their receivers start at $200, and it s at least $350 to integrate an existing stereo into the system. I d be shelling out over $1000 to use Sonos for this setup. As it is, I ve bought less than $100 of equipment (a second Pi and its accessories) and getting output that, while no doubt not quite as pristine, it still quite nice and acceptable. Quite nice. Update: LMS with Apache It turns out that Logitech Media Server can automatically save created playlists to a spot where Ampache can find them. So an Ampache streaming player can be used to access the same collection LMS uses, with full features. Nice!

1 January 2015

Russ Allbery: 2014 Book Reading in Review

This year, after a series of catastrophically horrible and unethical management decisions, I walked away from my job of seventeen years and found a new job. As you might expect, reading wasn't the top priority for much of the year. I'm moderately surprised that I read as much as I did. The good side is that I'm now in a much better place both professionally and personally and no longer have to put up with draining and demoralizing nonsense happening on a regular basis. The downside for my review output is that the new job is more engrossing and is, in some ways, harder work, so I expect my reading totals going forward to stabilize somewhere below where they were in the past (although it's possible that the daily commute will change that equation somewhat). As mentioned last year, I had a feeling that something like this would happen (although not that it would be anywhere near this bad), so I had no specific reading goals for the year. Next year, I'm going to see how it goes for the first few months, and might then consider setting some goals if I want to encourage myself to take more time for reading. The below statistics are confined to the books I reviewed in 2014. I read three more books that I've not yet reviewed, partly because the end of the year isn't as packed with vacation as it was at Stanford. Those will be counted in 2014. Despite the low reading totals for the year, I read two 10 out of 10 books. My favorite book of the year was Ann Leckie's Ancillary Justice, which was one of the best science fiction novels I've ever read. Highly recommended if you like the space opera genre at all. A close second was my favorite non-fiction book of the year and the other 10 out of 10: Allie Brosh's collection Hyperbole and a Half. Those of you who have read her blog already know her brilliant and insightful style of humor. Those who haven't are in for a treat. I read a lot of non-fiction this year and not as much fiction, partly for mood reasons, so I don't have honorable mentions in the fiction department. In the non-fiction department, though, there are four more books worth mentioning. Cryptography Engineering, by Niels Ferguson, Bruce Schneier, and Tadayoshi Kohno, was the best technical book that I read last year, and a must-read for anyone who works on security or crypto software. David Graeber's Debt was the best political and economic book of the year and the book from which I learned the most. It changed the way that you think about debt and loans significantly. A close second, though, was David Roodman's Due Diligence, which is a must-read for anyone who has considered investing in microfinance or is curious about the phenomenon. We need more data-driven, thoughtful, book-length analysis like this in the world. Finally, The Knowledge, by Lewis Dartnell, is an entertaining and quixotic project. The stated goal of the book is to document the information required to rebuild civilization after a catastrophe, with hopefully fewer false starts and difficult research than was required the first time. I'm dubious about its usefulness for that goal, but it's a fascinating and entertaining book in its own right, full of detail about industrial processes and the history of manufacturing and construction that are otherwise hard to come by without extensive (and boring) research. Recommended, even if you're dubious about the efficacy of the project. The full analysis includes some additional personal reading statistics, probably only of interest to me.

Tim Retout: Looking back at 2014

I have a tendency to forget what I've been up to - so I made a list for 2014. I started the year having recently watched many 30c3 videos online - these were fantastic, and I really should get round to the ones from 31c3. January is traditionally the peak time for the recruitment industry, so at work we were kept busy dealing with all the traffic. We'd recently switched the main job search to use Solr rather than MySQL, which helped - but we did spend a lot of time during the early months of the year converting tables from MyISAM to InnoDB. At the start of February was FOSDEM, and Kate and I took Sophie (then aged 10 months) to her first software conference. I grabbed a spot in the Go devroom for the Sunday afternoon, which was awesome. Downside: we got horribly ill while in Brussels. At work I was sorting out configuration management - this led to some Perl module backporting for Debian, and I uploaded Zookeeper at some point during the year as well. We currently make use of vagrant, chef and a combination of Debian packages and cpanm for Perl modules, but I have big plans to improve on that this year. Over a break from work I hacked up apt-transport-tor, which lets you install Debian packages over the Tor network. (This was inspired by videos from 30c3 and/or LibrePlanet, I think?) Continuing the general theme of paranoia, I attended the Don't Spy On Us campaign's day of action in June. Over the summer at work I was experimenting with Statsd and Graphite for monitoring. I also wrote Toggle, a Perl module for feature flags. In July I attended a London.pm meeting for the first time, and heard Thomas Klausner talk about OX - this nudged me into various talks at LPW (see below). Pubs have a lot to answer for. At some point I got an IPv6 tunnel working at home (although my ISP-provided router's wireless doesn't forward it), and I had an XBMC install going on a Raspberry Pi as another fun hack. In August and September I worked on packaging pump.io for Debian, and attended IndieWebCamp Brighton, where I delivered a talk/workshop on setting up TLS. (This all ties in to the paranoia theme.) I stalled the work on pump.io, partly because of licensing issues at build-dependency time (if you want to run all the tests) - but I expect I'll pick this up in 2015 once jessie is released. November was the London Perl Workshop, where I presented my work from the summer on statsd/graphite and Toggle, and a Bread::Board lightning talk. LPW was more enjoyable for me this year than previous years, probably because of the interesting people discussing various aspects of how feature flags ought to work. Simultaneously was the Cambridge MiniDebConf (why do these always clash?) where I think I fixed at least one RC bug. This is not an exhaustive list of everything I've done this year - there are more changes now lined up for 2015 which I haven't shared yet. But looking back, I'm pleased that the many small experiments I get up to do add up to something over time, and I can see that I'm achieving something. Here's to another year!

Next.

Previous.