Search Results: "tar"

30 September 2015

Chris Lamb: Free software activities in September 2015

Inspired by Rapha l Hertzog, here is a monthly update covering a large part of what I have been doing in the free software world:
Debian The Reproducible Builds project was also covered in depth on LWN as well as in Lunar's weekly reports (#18, #19, #20, #21, #22).
Uploads
  • redis A new upstream release, as well as overhauling the systemd configuration, maintaining feature parity with sysvinit and adding various security hardening features.
  • python-redis Attempting to get its Debian Continuous Integration tests to pass successfully.
  • libfiu Ensuring we do not FTBFS under exotic locales.
  • gunicorn Dropping a dependency on python-tox now that tests are disabled.



RC bugs


I also filed FTBFS bugs against actdiag, actdiag, bangarang, bmon, bppphyview, cervisia, choqok, cinnamon-control-center, clasp, composer, cpl-plugin-naco, dirspec, django-countries, dmapi, dolphin-plugins, dulwich, elki, eqonomize, eztrace, fontmatrix, freedink, galera-3, golang-git2go, golang-github-golang-leveldb, gopher, gst-plugins-bad0.10, jbofihe, k3b, kalgebra, kbibtex, kde-baseapps, kde-dev-utils, kdesdk-kioslaves, kdesvn, kdevelop-php-docs, kdewebdev, kftpgrabber, kile, kmess, kmix, kmldonkey, knights, konsole4, kpartsplugin, kplayer, kraft, krecipes, krusader, ktp-auth-handler, ktp-common-internals, ktp-text-ui, libdevice-cdio-perl, libdr-tarantool-perl, libevent-rpc-perl, libmime-util-java, libmoosex-app-cmd-perl, libmoosex-app-cmd-perl, librdkafka, libxml-easyobj-perl, maven-dependency-plugin, mmtk, murano-dashboard, node-expat, node-iconv, node-raw-body, node-srs, node-websocket, ocaml-estring, ocaml-estring, oce, odb, oslo-config, oslo.messaging, ovirt-guest-agent, packagesearch, php-svn, php5-midgard2, phpunit-story, pike8.0, plasma-widget-adjustableclock, plowshare4, procps, pygpgme, pylibmc, pyroma, python-admesh, python-bleach, python-dmidecode, python-libdiscid, python-mne, python-mne, python-nmap, python-nmap, python-oslo.middleware, python-riemann-client, python-traceback2, qdjango, qsapecng, ruby-em-synchrony, ruby-ffi-rzmq, ruby-nokogiri, ruby-opengraph-parser, ruby-thread-safe, shortuuid, skrooge, smb4k, snp-sites, soprano, stopmotion, subtitlecomposer, svgpart, thin-provisioning-tools, umbrello, validator.js, vdr-plugin-prefermenu, vdr-plugin-vnsiserver, vdr-plugin-weather, webkitkde, xbmc-pvr-addons, xfsdump & zanshin.

Norbert Preining: 6 years in Japan

Exactly 6 years ago, on October 1, 2009, I started my work at the Japan Advanced Institute of Science and Technology (JAIST), arriving the previous day in a place not completely unknown, but with a completely different outlook: I had a position as Associate Professor, and somehow was looking forward to an interesting and challenging time. japan-flag 6 years later I am still here at the JAIST, but things have changed considerably, and my future is even less clear than 6 years ago. So it is time to reflect a bit about the last years. The biggest achievement My biggest achievement in these 6 years is probably that I managed to learn Japanese to a degree that I can teach in Japanese (math, logic, etc), can read Japanese books to a certain degree, and have generally no problem communicating in daily life. Said that, there is still a long way to go. Reading, and much more writing, is still requiring concentration and power, far from the natural flow in my other languages. While talking feels rather natural, the complexity of the written language is a huge hurdle. But this is probably the good, the high point of the 6 years, a great challenge, that keeps my mind busy and working and challenged over long time, with still more to do. The happiest thing Many events here in Japan were of great fun and enjoyment for me. The rich culture, paired with a spectacular love for traditional handicraft I haven t seen anywhere else, is a guarantee for enjoyable and intellectually stimulating activities. But the biggest joy of my time here of course was that I found a lovely, beautiful, and caring wife. Not knowing the challenges of an international marriage, I was caught without preparation, and so we had (and still have) rough times due to the cultural differences, and different expectations. But this is what makes life interesting, and so I am always grateful for this chance. Whatever happens in the future, she will be part of my decisions and the center of my life. The biggest disappointment Of course, when you live in a country for some time, you learn to know the highs and lows. As someone interested in politics and social systems, Japan is a pain in the butt in many respects. But the biggest disappointment was in a different area: Working environment. While I love my work and had great surroundings, there is something that always is present in the background: Foreigners here are not considered assets, but embellishment. Meaning that they are the first ones to loose their jobs when times are difficult, meaning that they are not considered as full members. After many years at a university here, and with no outlook on a job after March, I can only say, Japan is a country of Japanese first , especially when it comes to jobs. Of course, other countries are not that different, but looking at the average mixture of nationalities at universities in Europe or the US, and comparing them to Japanese universities, a bleak image is arising. I enjoyed my time here, I worked hard and did a lot for my university, but the economically hard times make it necessary to change things, and that means getting rid of foreigners. That is the reason why the work environment is the biggest disappointment in these years. Future The future is unclear, as it always was. The dire fate of many researchers. Being in my 40ies without a permanent position and a family, I am forced to think hard what my next options are. The hide-and-seek games of Japanese (and other) universities seem to me less and less an option. Sad as it is, after having worked 20+ years in academics, having done some interesting (for me) research and having managed to secure a name in our community, I am not sure where my future is. Continuing on definite contracts does not sound like a great option for me. Several things for the future come to my mind: starting my own business, work as programmer (maybe Google still wants me after I rejected them 2 years ago), work as mountain guide (have done that for some years before going to Japan). All of that is possible, but my loosing the time to research will always be a pain, since I enjoy cracking my brain on some complicated and deep logical problems. Whatever comes, I will take it as a chance to learn new things. And in one way or another it will work out, I hope.

Raphaël Hertzog: My Free Software Activities in September 2015

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I have been paid to work 8 hours on Debian LTS. In that time, I mostly did CVE triaging (in the last 3 days since I m of LTS frontdesk duty this week). I pushed 14 commits to the security tracker. There were multiple CVE without any initial investigation so I checked the status of the CVE not only in squeeze but also in wheezy/jessie. On unpaid time, I wrote and sent the summary of the work session held during DebConf. And I tried to initiate a discussion about offering mysql-5.5 in squeeze-lts. We also have setup lts-security@debian.org so that we can better handle embargoed security updates. The Debian Administrator s Handbook Debian Handbook: cover of the jessie editionI spent a lot of time on my book, the content update has been done but now we re reviewing it before preparing the paperback. I also started updating its French translation. You can help review it too. While working on the book I noticed that snort got removed from jessie and the SE linux reference policy as well. I mailed their maintainers to recommend that they provide them in jessie-backports at least those packages are relatively important/popular and it s a pity that they are missing in jessie. I hope to finish the book update in the next two weeks! Distro Tracker I spent a lot of time to revamp the mail part of Distro Tracker. But as it s not finished yet, I don t have anything to show yet. That said I pushed an important fix concerning the mail subscriptions (see #798555), basically all subscriptions of packages containing a dash were broken. It just shows that the new tracker is not yet widely used for mail subscription I also merged a patch from Andrew Starr-Bochicchio (#797633) to improve the description of the WNPP action items. And I reviewed another patch submitted by Orestis Ioannou to allow browsing of old news (see #756766). And I filed #798011 against bugs.debian.org to request that a new X-Debian-PR-Severity header field be added to outgoing BTS mail so that Distro Tracker can filter mails by severity and offer people to subscribe to RC bugs only. Misc Debian work I filed many bugs this month and almost all of them are related to my Kali work: Thanks See you next month for a new summary of my activities.

3 comments Liked this article? Click here. My blog is Flattr-enabled.

Dominique Dumont: Using custom cache object with AngularJS $http

Hello At work, I ve been bitten by the way AngularJS handles cache by default when using $https service. This post will show a simple way to improve cache handling with $http service. The service I m working on must perform the followings tasks: At first, I ve naively used $http.get cache parameter to enable or disable caching using a sequence like:
  1. $http.get(url, cache: true )
  2. $http.post(url)
  3. $http.get(url, cache: false )
  4. $http.get(url, cache: true )
Let s say the calls above use the following data:
  1. $http.get(url, cache: true ) returns foo
  2. $http.post(url) stores bar
  3. $http.get(url, cache: false ) returns bar
I expected the next call $http.get(url, cache: false ) to return bar . But no, I got foo , i.e. the obsolete data. Turns out that cache object is completely left alone when cache: false is passed to $http.get. ok. Fair enough. But this means that the value of the cache parameter should not change for a given URL. The default cache provided by $https cannot be cleared. (Well, actually, you can clear the cache under AngularJS s hood, but that will probably not improve the readability of your code). The naive approach does not work. Let s try another solution by using a custom cache object as suggested by AngularJS doc. This cache object should be created by $cacheFactory service. This cache object can then be passed to $http.get to be used as cache. When needed, the cache can be cleared. In the example above, the cache must be cleared after saving some data to the remote service. There s 2 possibilities to clear a cache: So, we have to use the first solution and create a cache object for each API entry point:
angular.module('app').factory('myService', function ($http, $cacheFactory)  
  var myFooUrl = '/foo-rest-service';
  
  // create cache object. The cache id must be unique
  var fooCache = $cacheFactory('myService.foo'); 
  function getFooData ()  
    return $http.get( myFooUrl,   cache: fooCache  );
   ;
 
  function saveFooData(data)  
    return $http.post( myFooUrl,   cache: fooCache  ).then(function()  
      myCache.removeAll() ;
     );
   
 );
The code above ensures that: This simple approach has the following limitations: If you need more a more advance cache mechanism, you may want to check jmdobry s angular cache project All the best

29 September 2015

Dariusz Dwornikowski: Delete until signature in vim

It has been bugging me for a while. When responding to an email, you often want to delete all the content (or part of the previous content) until the end of the email's body. However it would be nice to leave your signature in place. For that I came up with this nifty little vim trick:
nnoremap <silent> <leader>gr <Esc>d/--\_.*Dariusz<CR>:nohl<CR>O
Assuming that your signature starts with -- and the following line starts with your name (in my case it is Dariusz), this will delete all the content from the current line until the signature. Then it will remove search highlighting, and finally move one line up.

28 September 2015

Antoine Beaupr : Fun experiments with laptop battery

After reading up on a eye-opening blog from Petter Reinholdtsen about laptop batteries, or more specifically Lithium-Ion laptop batteries, I figured I needed to try out the "TLP Linux Advanced Power Management" that I had been keeping an eye on for a while. tlp is yet another tool to control power usage on laptops (mostly Thinkpad, mine is a X120e). The novelty of tlp is the "hands off" approach: everything should be automatically configured for you... Obviously, that means I then went on working for a few hours on breaking and fixing my laptop in random operations. I opened a bunch of pull requests on the interesting battery status package that Petter produced to make it work with my setup and make it display graphs directly (instead of into a file). I also rewrote the graphing tool in Python with SciPy in order to have cleaner labels and be able to deduce the date at which a battery would be completely unusuable because it can't recharge high enough. (At the time of writing, the battery estimated death time is december 7th, but that data is skewed because of a quick change in the battery charge after the BIOS upgrade, below.) I then went on to try to limit my laptop charging to 80%, since this seems to make the battery last longer (sources from Petter: 1, 2, 3). Unfortunately, even after building a local (and trivial) backport of the tlp package to Debian stable (8.2/Jessie), I still couldn't access those controls, as TLP is really just a set of shell scripts that glue a bunch of stuff together.
The backport was simply:
apt-get source tlp
cd tlp*/
debuild
sudo dpkg -i ../tlp*.deb
I read here and there (and in Petter's post) that I needed the tp-smapi-dkms package, so I went ahead an installed it:
sudo apt install tp-smapi-dkms
(Yes, Jessie has a neat apt command now, it's great, upgrade now.) Unfortunately, this still didn't work. I think the error back then was something like:
thinkpad_ec: thinkpad_ec_request_row: arg0 rejected: (0x01:0x00)->0x00
thinkpad_ec: thinkpad_ec_read_row: failed requesting row: (0x01:0x00)->0xfffffffb
thinkpad_ec: initial ec test failed
I have seen suggestions here and there to try the acpi-call-dkms package, but that was useless as it doesn't support my model (but may work with others!). The error there was:
acpi_call: Cannot get handle: Error: AE_NOT_FOUND
Note: I still have it installed - it's unclear what impact it has, and I do not want to break my current setup.
So I then started to look at upgrading my BIOS, for some reason. I was running version 1.13 (8FET29WW) from 05/06/2011. I was able to update to 1.17 (8FET33WW) from 11/07/2012, using the memdisk binary from the syslinux-common package, with some help from the quite useful grub-imageboot package:
sudo apt install syslinux-common grub-imageboot
wget https://download.lenovo.com/ibmdl/pub/pc/pccbbs/mobiles/8fuj10uc.iso
sudo mkdir /boot/images
sudo mv 8fuj10uc.iso /boot/images
sudo reboot
I found the image on the Thinkpad x120e support page from Lenovo (which happily bounces around, so don't rely on the above URLs too much). When I rebooted, I was offered to boot from the image by grub, which went on fine, considering it was running some version of DOS, which is always a bit scary considering it is software that is somewhat almost as old as me.
I wish I could have installed some free software in the BIOS instead of the outdated crap that Lenovo provides, but unfortunately, it seems this will never be possible with Libreboot or Coreboot, mostly because Intel is evil and installs backdoors in all their computers. Fun times.
Fortunately and surprisingly, the update worked and went on pretty smoothly. After that, I was able to set the charge limit with:
echo 80   sudo tee /sys/devices/platform/smapi/BAT0/stop_charge_thresh
Amazing! I had almost forgotten why I almost bricked my system on a thursday, good thing that worked! Now the fun part was that, after some reboots or something I did, I am not sure what, the above stopped working: I couldn't load the drivers at all anymore, dmesg treating me with a nasty:
thinkpad_ec: thinkpad_ec_read_row: failed requesting row: (0x01:0x00)->0xfffffff0
thinkpad_ec: initial ec test failed
Now that is some obscure error message material! Fun stuff. I tried uninstalling tlp, the smapi modules, the acpi-call modules, rebooting, turning the machine off, removing the battery, nothing worked. Even more hilarious, the charge controler was now stuck at 80%: I had artificially destroyed 20% of the battery capacity in software. Ouch. I think this may have been related to uninstalling the tp-smapi-dkms package without unloading it at first. I found some weird entries in my kern.log like this:
tp_smapi unloaded.
thinkpad_ec: thinkpad_ec_read_row: failed requesting row: (0x14:0x00)->0xfffffff0
hdaps: cannot power off
hdaps: driver unloaded.
thinkpad_ec: unloaded.
I think that after that point, I couldn't load either module, not even thinkpad_ec... After tearing out a few more hairs and hammering my head on the keyboard randomly, I thought I could just try another BIOS upgrade, just for the fun of it. Turns out you actually can't rerun the upgrade, but you can change the model number through the same software, and this seems to reset some stuff. So I went back in the ISO image I had loaded earlier, and went on to change the model number (actually setting it to the same value, but whatever, it still ran the update). It turns out this seems to have reset a bunch of stuff and now everything works. I can use tlp setcharge and all the neat tools go well. The two key commands are:
# limit charging to 80% of the battery, but not lower than 40%
sudo tlp setcharge 40 80
# clear the above setting and just charge the battery to 100%
sudo tlp fullcharge
It seems that the 40% bit isn't supported by my laptop, but whatever: the battery stays charged when on AC power anyways, so I don't really understand what the setting is for in the first place. The error there is:
smapi smapi: smapi_request: SMAPI error: Function is not supported by SMAPI BIOS (func=2116)
smapi smapi: __get_real_thresh: cannot get start_thresh of bat=0: Function is not supported by SMAPI BIOS
The second command is what I will need to remember to run before I unplug the laptop for a trip. I suspect this will be really annoying and I may end up disabling all this stuff and just yank the power cable out when the battery reaches 80%, by hand, when I need to. But it was a fun geeking out, and hopefully this will be useful for others. And of course, the graphs from Petter will be interesting in a few months... Before the BIOS upgrade, the battery capacity was reported as 100% (actually, at 100.03%, which was strange). Now, the capacity is at 98.09%, which is probably just a more accurate reading that was fixed in the BIOS upgrade. Finally, also see the useful thinkwiki troubleshooting page and especially their interesting BIOS upgrade documentation which inspired me to write my own version. I would have gladly contributed to theirs, but I seem to have lost my password on this site, with no recovery possible... The arch linux wiki has obviously excellent documentation as well, as usual.

27 September 2015

Sven Hoexter: 1blu hack and the usual TLS certificate key madness

Some weeks ago the german low cost hoster 1blu got hacked and there was a bit of fuss later about the TLS certificates issued by 1blu. I think they reissued all of them. Since I knew that some hoster offer to generate the complete cert + key package for the customer I naively assumed that only the lazy and novice customers were the victims of that issue. Today, while helping someone, I learned that 1blu forces you to use the key generated by them for certificates included in a virtual server bundle and probably other bundles. That makes those bundles a lot less attractive since the included certificate is not useful at all. One could of course argue that a virtual server is not trustworthy anyway, but I'd like to believe for now that it's more complicated to extract stuff from all running virtual servers compared to dumping the central database / key repository. Maybe it's time to create a wrapper around openssl that is less opaque to novice users so we can get rid of key generation by a third party one day. In the end it's a disasterous trend that only got started because of usability issues.

Ben Armstrong: Annual Bluff Hike, 2015

Here is a photo journal of our hike on the Bluff Wilderness Trail with my friend, Ryan Neily, as is our tradition at this time of year. Rather than hike all four loops, as we achieved last year, we chose to cover only the Pot Lake and Indian Hill loops. Like our meandering pace, our conversations were enjoyable and far ranging, with Nature doing her part, stimulating our minds and bodies and refreshing our spirits.
A break at the summit of Pot Lake loop. Click to start slideshow.A break at the summit of Pot Lake loop. Click to start slideshow.
Northern bayberry A few showers quickly dissipated into light mist on the first leg of the hike Ryan, enjoying one of the many beautiful views Cormorant or shag. Hard to say from this poor, zoomed cellphone shot. Darkened pool amongst the rugged trees Late summer colours A riot of life shoots up in every crevice Large boulders and trees, forming a non-concrete alley along the trail margin Huckleberries still plentiful on the Indian Hill loop Sustenance to keep us going Not at all picked over, like the Pot Lake loop We break here for lunch Just about ready to embark on the last half We are surprised by the productivity of these short, scrubby huckleberries Barely rising from the reindeer moss, each huckleberry twig provides sweet, juicy handfuls A small pond on the trip back A break on the home stretch Common juniper, which nevertheless is not so common out here Immature green common juniper berries (actually cones)
facebooktwittergoogle_plusredditpinterestlinkedinmail

Lunar: Reproducible builds: week 22 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Packages fixed The following 22 packages became reproducible due to changes in their build dependencies: breathe, cdi-api, geronimo-jpa-2.0-spec, geronimo-validation-1.0-spec, gradle-propdeps-plugin, jansi, javaparser, libjsr311-api-java, mac-widgets, mockito, mojarra, pastescript, plexus-utils2, powerline, python-psutil, python-sfml, python-tldap, pythondialog, tox, trident, truffle, zookeeper. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: diffoscope development The changes to make diffoscope run under Python 3, along with many small fixes, entered the archive with version 35 on September 21th. Another release was made the very next day fixed two encoding-related issues discovered when running diffoscope on more Debian packages. strip-nondeterminism development Version 0.12.0 now preserves file permissions on modified zip files and dh_strip_nondeterminism has been made compatible with older debhelper. disorderfs development Version 0.3.0 implemented a multi-user mode that was required to build Debian packages using disorderfs. It also added command line options to control the ordering of files in directory (either shuffled or reversed) and another to do arbitrary changes to the reported space used by files on disk. A couple days later, version 0.4.0 was released to support locks, flush, fsync, fsyncdir, read_buf, and write_buf. Almost all known issues have now been fixed. reproducible.debian.net disorderfs is now used during the second build. This makes file ordering issue very easy to identify as such. (h01ger) Work has been done on making the distributed build setup more reliable. (h01ger) Documentation update Matt Kraii fixed the example on how to fix issues related to dates in Sphinx. Recent Sphinx versions should also be compatible with SOURCE_DATE_EPOCH. Package reviews 53 reviews have been removed, 85 added and 13 updated this week. 46 packages failing to build from source has been identified by Chris Lamb, Chris West, and Niko Tyni. Chris Lamb was the lucky reporter of bug #800000 on vdr-plugin-prefermenu. Issues related to disorderfs are being tracked with a new issue.

25 September 2015

Steve McIntyre: Linaro VLANd v0.4

VLANd is a python program intended to make it easy to manage port-based VLAN setups across multiple switches in a network. It is designed to be vendor-agnostic, with a clean pluggable driver API to allow for a wide range of different switches to be controlled together. There's more information in the README file. I've just released v0.4, with a lot of changes included since the last release: VLANd is Free Software, released under the GPL version 2 (or any later version). For now, grab it from git; tarballs will be coming shortly.

24 September 2015

Petter Reinholdtsen: The life and death of a laptop battery

When I get a new laptop, the battery life time at the start is OK. But this do not last. The last few laptops gave me a feeling that within a year, the life time is just a fraction of what it used to be, and it slowly become painful to use the laptop without power connected all the time. Because of this, when I got a new Thinkpad X230 laptop about two years ago, I decided to monitor its battery state to have more hard facts when the battery started to fail. First I tried to find a sensible Debian package to record the battery status, assuming that this must be a problem already handled by someone else. I found battery-stats, which collects statistics from the battery, but it was completely broken. I sent a few suggestions to the maintainer, but decided to write my own collector as a shell script while I waited for feedback from him. Via a blog post about the battery development on a MacBook Air I also discovered batlog, not available in Debian. I started my collector 2013-07-15, and it has been collecting battery stats ever since. Now my /var/log/hjemmenett-battery-status.log file contain around 115,000 measurements, from the time the battery was working great until now, when it is unable to charge above 7% of original capacity. My collector shell script is quite simple and look like this:
#!/bin/sh
# Inspired by
# http://www.ifweassume.com/2013/08/the-de-evolution-of-my-laptop-battery.html
# See also
# http://blog.sleeplessbeastie.eu/2013/01/02/debian-how-to-monitor-battery-capacity/
logfile=/var/log/hjemmenett-battery-status.log
files="manufacturer model_name technology serial_number \
    energy_full energy_full_design energy_now cycle_count status"
if [ ! -e "$logfile" ] ; then
    (
	printf "timestamp,"
	for f in $files; do
	    printf "%s," $f
	done
	echo
    ) > "$logfile"
fi
log_battery()  
    # Print complete message in one echo call, to avoid race condition
    # when several log processes run in parallel.
    msg=$(printf "%s," $(date +%s); \
	for f in $files; do \
	    printf "%s," $(cat $f); \
	done)
    echo "$msg"
 
cd /sys/class/power_supply
for bat in BAT*; do
    (cd $bat && log_battery >> "$logfile")
done
The script is called when the power management system detect a change in the power status (power plug in or out), and when going into and out of hibernation and suspend. In addition, it collect a value every 10 minutes. This make it possible for me know when the battery is discharging, charging and how the maximum charge change over time. The code for the Debian package is now available on github. The collected log file look like this:
timestamp,manufacturer,model_name,technology,serial_number,energy_full,energy_full_design,energy_now,cycle_count,status,
1376591133,LGC,45N1025,Li-ion,974,62800000,62160000,39050000,0,Discharging,
[...]
1443090528,LGC,45N1025,Li-ion,974,4900000,62160000,4900000,0,Full,
1443090601,LGC,45N1025,Li-ion,974,4900000,62160000,4900000,0,Full,
I wrote a small script to create a graph of the charge development over time. This graph depicted above show the slow death of my laptop battery. But why is this happening? Why are my laptop batteries always dying in a year or two, while the batteries of space probes and satellites keep working year after year. If we are to believe Battery University, the cause is me charging the battery whenever I have a chance, and the fix is to not charge the Lithium-ion batteries to 100% all the time, but to stay below 90% of full charge most of the time. I've been told that the Tesla electric cars limit the charge of their batteries to 80%, with the option to charge to 100% when preparing for a longer trip (not that I would want a car like Tesla where rights to privacy is abandoned, but that is another story), which I guess is the option we should have for laptops on Linux too. Is there a good and generic way with Linux to tell the battery to stop charging at 80%, unless requested to charge to 100% once in preparation for a longer trip? I found one recipe on askubuntu for Ubuntu to limit charging on Thinkpad to 80%, but could not get it to work (kernel module refused to load). I wonder why the battery capacity was reported to be more than 100% at the start. I also wonder why the "full capacity" increases some times, and if it is possible to repeat the process to get the battery back to design capacity. And I wonder if the discharge and charge speed change over time, or if this stay the same. I did not yet try to write a tool to calculate the derivative values of the battery level, but suspect some interesting insights might be learned from those. Update 2015-09-24: I got a tip to install the packages acpi-call-dkms and tlp (unfortunately missing in Debian stable) packages instead of the tp-smapi-dkms package I had tried to use initially, and use 'tlp setcharge 40 80' to change when charging start and stop. I've done so now, but expect my existing battery is toast and need to be replaced. The proposal is unfortunately Thinkpad specific.

Joachim Breitner: The Incredible Proof Machine

In a few weeks, I will have the opportunity to offer a weekend workshop to selected and motivated high school students1 to a topic of my choice. My idea is to tell them something about logic, proofs, and the joy of searching and finding proofs, and the gratification of irrevocable truths. While proving things on paper is already quite nice, it is much more fun to use an interactive theorem prover, such as Isabelle, Coq or Agda: You get immediate feedback, you can experiment and play around if you are stuck, and you get lots of small successes. Someone2 once called interactive theorem proving the worlds most geekiest videogame . Unfortunately, I don t think one can get high school students without any prior knowledge in logic, or programming, or fancy mathematical symbols, to do something meaningful with a system like Isabelle, so I need something that is (much) easier to use. I always had this idea in the back of my head that proving is not so much about writing text (as in normally written proofs) or programs (as in Agda) or labeled statements (as in Hilbert-style proofs), but rather something involving facts that I have proven so far floating around freely, and way to combine these facts to new facts, without the need to name them, or put them in a particular order or sequence. In a way, I m looking for labVIEW wrestled through the Curry-Horward-isomorphism. Something like this:
A proof of implication currying

A proof of implication currying

So I set out, rounded up a few contributors (Thanks!), implemented this, and now I proudly present: The Incredible Proof Machine3 This interactive theorem prover allows you to do perform proofs purely by dragging blocks (representing proof steps) onto the paper and connecting them properly. There is no need to learn syntax, and hence no frustration about getting that wrong. Furthermore, it comes with a number of example tasks to experiment with, so you can simply see it as a challenging computer came and work through them one by one, learning something about the logical connectives and how they work as you go. For the actual workshop, my plan is to let the students first try to solve the tasks of one session on their own, let them draw their own conclusions and come up with an idea of what they just did, and then deliver an explanation of the logical meaning of what they did. The implementation is heavily influenced by Isabelle: The software does not know anything about, say, conjunction ( ) and implication ( ). To the core, everything is but an untyped lambda expression, and when two blocks are connected, it does unification4 of the proposition present on either side. This general framework is then instantiated by specifying the basic rules (or axioms) in a descriptive manner. It is quite feasible to implement other logics or formal systems on top of this as well. Another influence of Isabelle is the non-linear editing: You neither have to create the proof in a particular order nor have to manually manage a proof focus . Instead, you can edit any bit of the proof at any time, and the system checks all of it continuously. As always, I am keen on feedback. Also, if you want to use this for your own teaching or experimenting needs, let me know. We have a mailing list for the project, the code is on GitHub, where you can also file bug reports and feature requests. Contributions are welcome! All aspects of the logic are implemented in Haskell and compiled to JavaScript using GHCJS, the UI is plain hand-written and messy JavaScript code, using JointJS to handle the graph interaction. Obviously, there is still plenty that can be done to improve the machine. In particular, the ability to create your own proof blocks, such as proof by contradiction, prove them to be valid and then use them in further proofs, is currently being worked on. And while the page will store your current progress, including all proofs you create, in your browser, it needs better ways to save, load and share tasks, blocks and proofs. Also, we d like to add some gamification, i.e. achievements ( First proof by contradiction , 50 theorems proven ), statistics, maybe a share theorem on twitter button. As the UI becomes more complicated, I d like to investigating moving more of it into Haskell world and use Functional Reactive Programming, i.e. Ryan Trickle s reflex, to stay sane. Customers who liked The Incredible Proof Machine might also like these artifacts, that I found while looking whether something like this exists:

  1. Students with migration background supported by the START scholarship
  2. Does anyone know the reference?
  3. We almost named it Proofcraft , which would be a name our current Minecraft-wild youth would appreciate, but it is alreay taken by Gerwin Kleins blog. Also, the irony of a theorem prover being in-credible is worth something.
  4. Luckily, two decades ago, Tobias Nipkow published a nice implementation of higher order pattern unification as ML code, which I transliterated to Haskell for this project.

23 September 2015

Simon Josefsson: Cosmos A Simple Configuration Management System

Back in early 2012 I had been helping with system administration of a number of Debian/Ubuntu-based machines, and the odd Solaris machine, for a couple of years at $DAYJOB. We had a combination of hand-written scripts, documentation notes that we cut n paste d from during installation, and some locally maintained Debian packages for pulling in dependencies and providing some configuration files. As the number of people and machines involved grew, I realized that I wasn t happy with how these machines were being administrated. If one of these machines would disappear in flames, it would take time (and more importantly, non-trivial manual labor) to get its services up and running again. I wanted a system that could automate the complete configuration of any Unix-like machine. It should require minimal human interaction. I wanted the configuration files to be version controlled. I wanted good security properties. I did not want to rely on a centralized server that would be a single point of failure. It had to be portable and be easy to get to work on new (and very old) platforms. It should be easy to modify a configuration file and get it deployed. I wanted it to be easy to start to use on an existing server. I wanted it to allow for incremental adoption. Surely this must exist, I thought. During January 2012 I evaluated the existing configuration management systems around, like CFEngine, Chef, and Puppet. I don t recall my reasons for rejecting each individual project, but needless to say I did not find what I was looking for. The reasons for rejecting the projects I looked at ranged from centralization concerns (single-point-of-failure central servers), bad security (no OpenPGP signing integration), to the feeling that the projects were too complex and hence fragile. I m sure there were other reasons too. In February I started going back to my original needs and tried to see if I could abstract something from the knowledge that was in all these notes, script snippets and local dpkg packages. I realized that the essence of what I wanted was one shell script per machine, OpenPGP signed, in a Git repository. I could check out that Git repository on every new machine that I wanted to configure, verify the OpenPGP signature of the shell script, and invoke the script. The script would do everything needed to get the machine up into an operational stage again, including package installation and configuration file changes. Since I would usually want to modify configuration files on a system even after its initial installation (hey not everyone is perfect), it was natural to extend this idea to a cron job that did git pull , verified the OpenPGP signature, and ran the script. The script would then have to be a bit more clever and not redo everything every time. Since we had many machines, it was obvious that there would be huge code duplication between scripts. It felt natural to think of splitting up the shell script into a directory with many smaller shell scripts, and invoke each shell script in turn. Think of the /etc/init.d/ hierarchy and how it worked with System V initd. This would allow re-use of useful snippets across several machines. The next realization was that large parts of the shell script would be to create configuration files, such as /etc/network/interfaces. It would be easier to modify the content of those files if they were stored as files in a separate directory, an overlay stored in a sub-directory overlay/, and copied into the file system s hierarchy with rsync. The final realization was that it made some sense to run one set of scripts before rsync ing in the configuration files (to be able to install packages or set things up for the configuration files to make sense), and one set of scripts after the rsync (to perform tasks that require some package to be installed and configured). These set of scripts were called the pre-tasks and post-tasks respectively, and stored in sub-directories called pre-tasks.d/ and post-tasks.d/. I started putting what would become Cosmos together during February 2012. Incidentally, I had been using etckeeper on our machines, and I had been reading its source code, and it greatly inspired the internal design of Cosmos. The git history shows well how the ideas evolved even that Cosmos was initially called Eve but in retrospect I didn t like the religious connotations and there were a couple of rewrites on the way, but on the 28th of February I pushed out version 1.0. It was in total 778 lines of code, with at least 200 of those lines being the license boiler plate at the top of each file. Version 1.0 had a debian/ directory and I built the dpkg file and started to deploy on it some machines. There were a couple of small fixes in the next few days, but development stopped on March 5th 2012. We started to use Cosmos, and converted more and more machines to it, and I quickly also converted all of my home servers to use it. And even my laptops. It took until September 2014 to discover the first bug (the fix is a one-liner). Since then there haven t been any real changes to the source code. It is in daily use today. The README that comes with Cosmos gives a more hands-on approach on using it, which I hope will serve as a starting point if the above introduction sparked some interest. I hope to cover more about how to use Cosmos in a later blog post. Since Cosmos does so little on its own, to make sense of how to use it, you want to see a Git repository with machine models. If you want to see how the Git repository for my own machines looks you can see the sjd-cosmos repository. Don t miss its README at the bottom. In particular, its global/ sub-directory contains some of the foundation, such as OpenPGP key trust handling.

22 September 2015

Russ Allbery: Review: Half Life

Review: Half Life, by S.L. Huang
Series: Russell's Attic #2
Publisher: S.L. Huang
Copyright: 2014
ISBN: 0-9960700-5-2
Format: Kindle
Pages: 314
This is a sequel to Zero Sum Game and the second book about Cas Russell, a mercenary superhero (in a world without the concept of superheroes) with preternatural ability to analyze anything about her surroundings with mathematics. While it reuses some personal relationships from the first book and makes a few references to the villains, it's a disconnected story. It would be possible to start here if you wanted to. Cas is now in the strange and unexpected situation of having friends, and they're starting to complicate her life. First, Arthur has managed to trigger some unexpected storehouse of morals and gotten her to try to stop killing people on jobs. That conscience may have something to do with her willingness to take a job from an apparently crazy man who claims a corporation has stolen his daughter, a daughter who appears nowhere in any official records. And when her other friend, Checker, gets in trouble with the mob, Cas tries to protect him in her own inimitable way, which poses a serious risk of shortening her lifespan. Even more than the first book, the story in Half Life is a mix of the slightly ridiculous world of superheroes with gritty (and bloody) danger. It featuring hit men, armed guards, lots of guns, and quite a lot of physical injury and blood. A nasty corporation that's obviously hiding serious secrets shares pages with the matriarch of a mob family who considers Checker sleeping with her daughter to be an abuse of her honor. The story eventually escalates into more outlandish bits of technology, an uncanny little girl, and a plot that would feel at home in a Batman comic. I like books that don't take themselves too seriously, but the contrast between the brutal treatment Cas struggles through and the outrageous mad scientist villain provokes a bit of cognitive whiplash. That said, the villains of Half Life are neither as freakish nor as disturbing as those in Zero Sum Game, which I appreciated. Huang packs in several plot twists, some inobvious decisions and disagreements between Russell and her friends about appropriate strategy, and Cas's discovery that there are certain things she cares very strongly about other than money and having jobs. Cas goes from a barely moral, very dark hero in the first book to something closer to a very grumbly chaotic good who insists she's not as good as she actually is. It's a standard character type, but Huang does a good job with it. Huang also adds a couple of supporting cast members in this book that I hope will stick around. Pilar starts as a receptionist at one of the companies Cas breaks into, and at first seems like she might be comic relief. But she ends up being considerably more competent than she first appears (or that she seems to realize); by the end of the book, I had a lot of respect for her. And Miri makes only a few appearances, but her unflappable attitude is a delight. I hope to see more of her. The biggest drawback to this book for me is that Cas gets hurt a lot. At times, the story falls into one of the patterns of urban fantasy: the protagonist gets repeatedly beaten up and abused until they figure out what's going on, and spends most of the story up against impossible odds and feeling helpless. That's not a plot pattern I'm fond of. I don't enjoy reading about physical pain, and I had trouble at some points in the story with the constant feeling of dread. Parts of the book I read in short bursts, putting it aside to look at something else. But the sense of dread falls off towards the end of the book, as Cas figures out what's actually going on, and none of it is as horrible as it felt it could be. If you have a similar problem with some urban fantasy tropes, I think it's safe to stick with the story. This was a fun story, but it doesn't develop much in the way of deeper themes in the series. There's essentially no Rio, no further discoveries about the villains of the first book, and no further details on what makes Cas tick or why she seems to be the only, or at least one of the few, super-powered people in this world. The advance publicity for the third book seems to indicate that's coming next. I'm curious enough now that I'll keep reading this series. Recommended if you liked the first book. Half Life is very similar, but I think slightly better. Followed by Root of Unity. Rating: 7 out of 10

21 September 2015

Jonathan McDowell: Getting a Dell E7240 working with a dock + a monitor

I have a Dell E7240. I m pretty happy with it - my main complaint is that it has a very shiny screen, and that seems to be because it s the touchscreen variant. While I don t care about that feature I do care about the fact it means I get FullHD in 12.5 Anyway. I ve had issues with using a dock and an external monitor with the laptop for some time, including getting so far as mentioning the problems on the appropriate bug tracker. I ve also had discussions with a friend who has the same laptop with the same issues, and has some time trying to get it reliably work. However up until this week I haven t had a desk I m sitting at for any length of time to use the laptop, so it s always been low priority for me. Today I sat down to try and figure out if there had been any improvement. Firstly I knew the dock wasn t at fault. A Dell E6330 works just fine with multiple monitors on the same dock. The E6330 is Ivybridge, while the E7240 is Haswell, so I thought potentially there might be an issue going on there. Further digging revealed another wrinkle I hadn t previously been aware of; there is a DisplayPort Multi-Stream Transport (MST) hub in play, in particular a Synaptics VMM2320. Dell have a knowledge base article about Multiple external display issues when docked with a Latitude E7440/E7240 which suggests a BIOS update (I was already on A15) and a firmware update for the MST HUB. Sadly the firmware update is Windows only, so I had to do a suitable dance to be able to try and run it. I then discovered that the A05 update refused to work, complaining I had an invalid product ID. The A04 update did the same. The A01 update thankfully succeeded and told me it was upgrading from 2.00.002 to 2.15.000. After that had completed (and I d power cycled to switch to the new firmware) I tried A05 again and this time it worked and upgraded me to 2.22.000. Booting up Linux again I got further than before; it was definitely detecting that there was a monitor but it was very unhappy with lots of [drm:intel_dp_start_link_train] *ERROR* too many full retries, give up errors being logged. This was with 4.2, and as I d been meaning to try 4.3-rc2 I thought this was a good time to give it a try. Lo and behold, it worked! Even docking and undocking does what I d expect, with the extra monitor appearing / disappearing as you d expect. Now, I m not going to assume this means it s all happy, as I ve seen this sort-of work in the past, but the clue about MST, the upgrade of that firmware (and noticing that it made things better under Windows as well) and the fact that there have been improvements in the kernel s MST support according to the post 4.2 log gives me some hope that things will be better from here on.

Daniel Pocock: Skype outage? reSIProcate to the rescue!

On Friday, the reSIProcate community released the latest beta of reSIProcate 1.10.0. One of the key features of the 1.10.x release series is support for presence (buddy/status lists) over SIP, the very thing that is currently out of action in Skype. This is just more proof that free software developers are always anticipating users' needs in advance. reSIProcate 1.10.x also includes other cool things like support for PostgreSQL databases and Perfect Forward Secrecy on TLS. Real free software has real answers Unlike Skype, reSIProcate is genuine free software. You are free to run it yourself, on your own domain or corporate network, using the same service levels and support strategies that are important for you. That is real freedom. Not sure where to start? If you have deployed web servers and mail servers but you are not quite sure where to start deploying your own real-time communications system, please check out the RTC Quick Start Guide. You can read it online or download the PDF e-book. Is your community SIP and XMPP enabled? The Debian community has a federated SIP service, supporting standard SIP and WebRTC at rtc.debian.org for all Debian Developers. XMPP support was tested at DebConf15 and will be officially announced very soon now. A similar service has been developed for the Fedora community and it is under evaluation at fedrtc.org. Would you like to extend this concept to other free software and non-profit communities that you are involved in? If so, please feel free to contact me personally for advice about how you can replicate these successful initiatives. If your community has a Drupal web site, then you can install everything using packages and the DruCall module. Comment and discuss Please join the Free-RTC mailing list to discuss or comment

Lunar: Reproducible builds: week 21 in Stretch cycle

If you see someone on the Debian ReproducibleBuilds project, buy him/her a beer. This work is awesome. What happened in the reproducible builds effort this week: Media coverage Nathan Willis covered our DebConf15 status update in Linux Weekly News. Access to non-LWN subscribers will be given on Thursday 24th. Linux Journal published a more general piece last Tuesday. Unexpected praise for reproducible builds appeared this week in the form of several iOS applications identified as including spyware. The malware was undetected by Apple screening. This actually happened because application developers had simply downloaded a trojaned version of XCode through an unofficial source. While reproducible builds can't really help users of non-free software, this is exactly the kind of attacks that we are trying to prevent in our systems. Toolchain fixes Niko Tyni wrote and uploaded a better patch for the source order problem in libmodule-build-perl. Tristan Seligmann identified how the code generated by python-cffi could be emitted in random order in some cases. Upstream has already fixed the problem. Packages fixed The following 24 packages became reproducible due to changes in their build dependencies: apache-curator, checkbox-ng, gant, gnome-clocks, hawtjni, jackrabbit, jersey1, libjsr305-java, mathjax-docs, mlpy, moap, octave-geometry, paste, pdf.js, pyinotify, pytango, python-asyncssh, python-mock, python-openid, python-repoze.who, shadow, swift, tcpwatch-httpproxy, transfig. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: reproducible.debian.net Tests for Coreboot, OpenWrt, NetBSD, and FreeBSD now runs weekly (instead of monthly). diffoscope development Python 3 offers new features (namely yield from and concurrent.futures) that could help implement parallel processing. The clear separation of bytes and unicode strings is also likely to reduce encoding related issues. Mattia Rizolo thus kicked the effort of porting diffoscope to Python 3. tlsh was the only dependency missing a Python 3 module. This got quickly fixed by a new upload. The rest of the code has been moved to the point where only incompatibilities between Python 2.7 and Pyhon 3.4 had to be changed. The commit stream still require some cleanups but all tests are now passing under Python 3. Documentation update The documentation on how to assemble the weekly reports has been updated. (Lunar) The example on how to use SOURCE_DATE_EPOCH with CMake has been improved. (Ben Beockel, Daniel Kahn Gillmor) The solution for timestamps in man pages generated by Sphinx now uses SOURCE_DATE_EPOCH. (Mattia Rizzolo) Package reviews 45 reviews have been removed, 141 added and 62 updated this week. 67 new FTBFS reports have been filled by Chris Lamb, Niko Tyni, and Lisandro Dami n Nicanor P rez Meyer. New issues added this week: randomness_in_r_rdb_rds_databases, python-ply_compiled_parse_tables. Misc. The prebuilder script is now properly testing umask variations again. Santiago Villa started a discussion on debian-devel on how binNMUs would work for reproducible builds.

Norbert Preining: International Sad Hits Volume 1

I stumbled over this CD a few weeks ago, and immediately ordered it from some second-hand dealer in the US: International Sad Hits Volume 1: Altaic Group. Four artists from different countries (2x Japan, Korea, Turkey) and very different music style, but connected in one thing: That they don t fit into the happy-peppy culture of AKB48, JPOP, KPOP and the like, but singers and songwriters that probe the depths of sadness. With two of my favorite Japanese appearing in the list (Tomokawa Kazuki and Mikami Kan), there was no way I could not buy this CD. international-sad-hits-volume-1 The four artist combined in this excellent CD are: Fikret K z lok, a Turkish musician, singer and songwriter. Quoting from the pamphlet:
However, in 1983 K z lok returned with one of his most important and best albums: Zaman Zaman (Time to time). [ ] These albums presented an alternative to the horrible pop scene emerging in Turkey they criticized the political situation, the so-called intellectuals, and the pop stars.
The Korean artist is (Kim Doo Soo), who was the great surprise of the CD for me. The sadness and beauty that is transmitted through is music is special. The pamphlet of the CD states:
The strictness of his home atmosphere suffocated him, and in defiance against his father he dropped out, and walked along the roads. He said later that, Hatred towards the world, and the emptiness of life overwhelmed me. I lived my life with alcohol every day.
After some problems due to political crisis, and fierce reactions to his song Bohemian (included) made him disappear into the mountains for 10 years, only to return with even better music. The third artist is one of my big favorites, Tomokawa Kazuki (Official site, Wikipedia) from Japan. I have written often about Tomokawa, but found a very fitting description in the booklet:
Author Tatematsu Wahei has described Tomokawa as a man standing naked, his sensibility utterly exposed and tingling. It s an accurate response to a creativity that seems unmediated by embarrassment, voraciously feeding off the artist s personal concern.
The forth artist is again from Japan, Mikami Kan, a well known wild man from the Japanese music scene. After is debut at the Nakatsugawa All-Japan Folk Jamboree in 1971, he was something like a star for some years, but without a record deal his popularity decreased steadily.
During this period his songwriting gradually altered, becoming more dense, surreal and uncompromisingly personal.
Every song on this CD is a masterpiece by itself, but despite me being a great fan of Tomokawa, my favorite is Kim Doo Soo here, with songs that grip your heart and soul, stunningly beauty and sad at the same time.

20 September 2015

Matthew Garrett: The Internet of Incompatible Things

I have an Amazon Echo. I also have a LIFX Smart Bulb. The Echo can integrate with Philips Hue devices, letting you control your lights by voice. It has no integration with LIFX. Worse, the Echo developer program is fairly limited - while the device's built in code supports communicating with devices on your local network, the third party developer interface only allows you to make calls to remote sites[1]. It seemed like I was going to have to put up with either controlling my bedroom light by phone or actually getting out of bed to hit the switch.

Then I found this article describing the implementation of a bridge between the Echo and Belkin Wemo switches, cunningly called Fauxmo. The Echo already supports controlling Wemo switches, and the code in question simply implements enough of the Wemo API to convince the Echo that there's a bunch of Wemo switches on your network. When the Echo sends a command to them asking them to turn on or off, the code executes an arbitrary callback that integrates with whatever API you want.

This seemed like a good starting point. There's a free implementation of the LIFX bulb API called Lazylights, and with a quick bit of hacking I could use the Echo to turn my bulb on or off. But the Echo's Hue support also allows dimming of lights, and that seemed like a nice feature to have. Tcpdump showed that asking the Echo to look for Hue devices resulted in similar UPnP discovery requests to it looking for Wemo devices, so extending the Fauxmo code seemed plausible. I signed up for the Philips developer program and then discovered that the terms and conditions explicitly forbade using any information on their site to implement any kind of Hue-compatible endpoint. So that was out. Thankfully enough people have written their own Hue code at various points that I could figure out enough of the protocol by searching Github instead, and now I have a branch of Fauxmo that supports searching for LIFX bulbs and presenting them as Hues[2].

Running this on a machine on my local network is enough to keep the Echo happy, and I can now dim my bedroom light in addition to turning it on or off. But it demonstrates a somewhat awkward situation. Right now vendors have no real incentive to offer any kind of compatibility with each other. Instead they're all trying to define their own ecosystems with their own incompatible protocols with the aim of forcing users to continue buying from them. Worse, they attempt to restrict developers from implementing any kind of compatibility layers. The inevitable outcome is going to be either stacks of discarded devices speaking abandoned protocols or a cottage industry of developers writing bridge code and trying to avoid DMCA takedowns.

The dystopian future we're heading towards isn't Gibsonian giant megacorporations engaging in physical warfare, it's one where buying a new toaster means replacing all your lightbulbs or discovering that the code making your home alarm system work is now considered a copyright infringement. Is there a market where I can invest in IP lawyers?

[1] It also requires an additional phrase at the beginning of a request to indicate which third party app you want your query to go to, so it's much more clumsy to make those requests compared to using a built-in app.
[2] I only have one bulb, so as yet I haven't added any support for groups.

comment count unavailable comments

Russ Allbery: Review: Shady Characters

Review: Shady Characters, by Keith Houston
Publisher: W.W. Norton
Copyright: 2013
ISBN: 0-393-34972-1
Format: Trade paperback
Pages: 250
Subtitled The Secret Life of Punctuation, Symbols & Other Typographical Marks, Shady Characters is one of those delightfully quirky books that examines the history of something you would not normally connect to history. It's an attempt to document, and in some cases reconstruct, the history of several specific punctuation marks. If you've read and enjoyed Lynn Truss's Eats, Shoots & Leaves, this is a near-perfect complement, focusing on more obscure marks and adding in a more detailed and coherent history. Houston has some interest in the common and quotidian punctuation marks, with chapters on the hyphen, dash, and quotation mark, but he avoids giving a full-chapter treatment to the most obvious periods and commas. (Although one learns quite a bit about them as asides in other histories.) The rest of the book focuses on the newly-popular (the at symbol), the recognizable but less common (the hash mark, the asterisk and dagger symbols used for footnotes, and the ampersand), and the historical but now obscure (the pilcrow or paragraph mark, and the manicule or pointing finger). He even devotes two chapters to unsuccessful invented punctuation: the interrobang and the long, failed history of irony and sarcasm punctuation. And this is an actual history, not just a collection of anecdotes and curious facts. Houston does a great job of keeping the text light, engaging, and readable, but he's serious about his topic. There are many reproductions of ancient documents showing early forms of punctuation, several serious attempts to adjudicate between competing theories of origin, a few well-marked and tentative forays into guesswork, and an open acknowledgment of several areas where we simply don't know. Along the way, the reader gets a deeper feel for the chaos and arbitrary personal stylistic choices of the days of hand-written manuscripts and the transformation of punctuation by technology. So much of what we now use in punctuation was shaped and narrowed by the practicalities of typesetting. And then modern technology revived some punctuation, such as the now-ubiquitous at sign, or the hash mark that graces every telephone touchpad. I think my favorite part of this history was using punctuation as perspective from which to track the changing relationship between people and written material. It's striking how much early punctuation was primarily used for annotations and commentary on the text, as opposed to making the text itself more readable. Much early punctuation was added after the fact, and then slowly was incorporated into the original manuscript, first via recopying and then via intentional authorial choice. Texts started including their own pre-emptive commentary, and brought the corresponding marks along with the notes. And then printing forced a vast simplification and standardization of punctuation conventions. Full compliments to W.W. Norton, as well, for putting the time and effort into making the paper version of this book a beautiful artifact. Punctuation is displayed in red throughout when it is the focus of the text. Endnotes are properly split from footnotes, with asides and digressions presented properly at the bottom of the page, and numbered endnotes properly reserved solely for citations. There is a comprehensive index, list of figures, and a short and curated list of further readings. I'm curious how much of the typesetting care was preserved in the ebook version, and dubious that all of it would have been possible given the current state of ebook formatting. Typesetting, obscure punctuation, and this sort of popular history of something quotidian are all personal interests of mine, so it's unsurprising I found this book delightful. But it's so well-written and engaging that I can recommend it even to people less interested in those topics. The next time you're in the mood for learning more about a corner of human history that few people consider, I recommend Shady Characters to your attention. Rating: 8 out of 10

Next.

Previous.