Search Results: "rul"

3 March 2016

Tianon Gravi: Docker on VULTR + IPv6

I ve been using VULTR for a little while now and have been generally very pleased (especially with the very recent facelift the management portal received). I don t want to waste too much time talking about it, but the killer feature for me (over some of their competitors like DigitalOcean) is that I can provide a raw ISO and provision my VM directly using it as I would any local VM (which also means that once my VM is up and working, I get to use the OS s standard kernel, which is especially important for using Debian Unstable well). Anyhow, already too much about that let s get to the cool stuff. Getting right down to the beef, let s assume I ve got a VULTR instance already created, my OS is already installed and working, I ve enabled IPv6 within VULTR, ensured that my VM is able to ping6 google.com (to verify at least basic routability), and have Docker version 1.10.2 installed. For the sake of demonstration, we ll assume that VULTR has assigned my IPv6 as follows: (available under the VM details via Settings > IPv6) (The astute reader may recognize RFC3849 here. ) The relevant documentation which helped me get to the working state outlined below is in the IPv6 with Docker section. The first step I took was creating a systemd drop-in file so that I could modify the daemon startup parameters (to include --ipv6 and --fixed-cidr-v6):
# /etc/systemd/system/docker.service.d/ipv6.conf
[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon -H fd:// --ipv6 --fixed-cidr-v6 2001:db8::/80
I chose to use just /80 for Docker any other reasonable prefix (assuming it is routed to your host / host network) should also work; the documentation I linked above has an example using a /125, for example. With this half in place, I can systemctl daemon-reload and systemctl restart docker.service, and when I start a container it will be automatically assigned an IPv6 address from within that prefix. Excellent. An important caveat to note is that this will break discovery on our host due to Docker enabling forwarding for us, so (assuming your internet-facing interface is named ens3 for the sake of illustration; it might just as easily be eth0, eth1, enps3, lan0, wlan0, etc) I had to sysctl net.ipv6.conf.ens3.accept_ra=2, and I added it to /etc/sysctl.d/docker-ipv6.conf for good measure (so that I don t lose it after I reboot). The second half of our IPv6 to containers problem is routing. The nitty-gritty details of this are discussed in the Using NDP proxying section of the documentation, but the gist is that my containers have IPv6 addresses, but the outside world doesn t have a route that leads to them, and that we need to tell the kernel to respond to solicitations for our container s IPv6 addresses appropriately. The kernel has a mechanism for doing so (via ip -6 neigh ...), but it is limited to individual addresses and is thus not especially great for having a solution that works magically without further manual labor per-container. This is where ndppd (also packaged for Debian as ndppd) came in.
# /etc/ndppd.conf
proxy ens3  
	rule 2001:db8::/80  
	 
 
After getting this configuration in place and restarting ndppd (systemctl restart ndppd), magic happened. My containers could ping6 google.com, and my other IPv6 hosts could ping6 the IPv6 addresses of my individual containers! You ve probably noted that this configuration isn t exactly secure, since it means that each of my individual containers has a publicly routable IPv6 address, but for this specific use case, I m OK with that! Update (2015-03-09): thanks to for correcting my systemd drop-in file usage!

1 March 2016

Vincent Sanders: Hope is tomorrow's veneer over today's disappointment.

Recently I have been very hopeful about the 96boards Hikey SBC and as Evan Esar predicted I have therefore been very disappointed. I was given a Hikey after a Linaro connect event some time ago by another developer who could not get the system working usefully and this is the tale of what followed.
The Standard Design
This board design was presented as Linaro creating a standard for the 64bit Single Board Computer (SBC) market. So I had expected that a project with such lofty goals would have considered many factors and provided at least as good a solution as the existing 32bit boards.

The lamentable hubris of creating a completely new form factor unfortunately sets a pattern for the whole enterprise. Given the aim towards "makers" I would have accepted that the system would not be a ATX PC size motherboard, but mini/micro/nano and pico ITX have been available for several years.

If opting for a smaller "credit card" form factor why not use one of the common ones that have already been defined by systems such as the Raspberry Pi B+? Instead now every 96Board requires special cases and different expansion boards.

Not content with defining their own form factor the design also uses a 8-18V supply, this is the only SBC I own that is not fed from a 5V supply. I understand that a system might require more current than a micro USB connector can provide, but for example the Banana Pi manages with a DC barrel jack readily capable of delivering 25W which would seem more than enough

The new form factor forced the I/O connectors to be placed differently to other SBC, given the opportunity to concentrate all connectors on one edge (like ATX designs) and avoid the issues where two or three sides are used. The 96board design instead puts connectors on two edges removing any possible benefit this might have given.

The actual I/O connectors specified are rather strange. There is a mandate for HDMI removing the possibility of future display technology changes. The odd USB arrangement of two single sockets instead of a stacked seems to be an attempt to keep height down but the expansion headers and CPU heatsink mean this is largely moot.

The biggest issue though is mandating WIFI but not Ethernet (even as an option), everything else in the design I could logically understand but this makes no sense. It means the design is not useful for many applications without adding USB devices.

Expansion is presented as a 2mm pitch DIL socket for "low speed" signals and a high density connector for "high speed" signals. The positioning and arrangement of these connectors proffered an opportunity to improve upon previous SBC designs which was not taken. The use of 2mm pitch and low voltage signals instead of the more traditional 2.54mm pitch 3.3v signals means that most maker type applications will need adapting from the popular Raspberry Pi and Arduino style designs.

In summary the design appears to have been a Linaro project to favour one of their members which took a Hisilicon Android phone reference design and put it onto a board with no actual thought beyond getting it done. Then afterwards attempted to turn that into a specification, this has simply not worked as an approach.

My personal opinion is that this specification is fatally flawed and, is a direct cause of, the bizarre situation where the "consumer" specification exists alongside the "enterprise" edition which itself has an option of microATX form factor anyhow!
The ImplementationIf we ignore the specification appearing to be nothing more than a codification of the original HiKey design we can look at the HiKey as an implementation.

Initially the board required modifying to add headers to attach a USB to 1.8V LVTTL serial adaptor on the UART0 serial port. Once Andy Simpkins had made this change for me I was able to work through the instructions and attempt to install a bootloader and OS image.

The initial software was essentially HiSilicon vendor code using the Android fastboot system to configure booting. There was no source and the Ubuntu OS images were an obvious afterthought to the Android images. Just getting these images installed required a great deal of effort, repetition and debugging. It was such a dreadful experience this signalled the commencement one of the repeated hiatuses throughout this project, the allure of 64 bit ARM computing has its limits even for me.

When I returned to the project I attempted to use the system from the on-board eMMC but the pre-built binary only kernel and OS image were very limited. Building a replacement kernel , or even modules for the existing one proved fruitless and the system was dreadfully unstable.

I wanted to use the system as a builder for some Open Source projects but the system instability ruled this out. I considered attempting to use virtualisation which would also give better system isolation for builder systems. By using KVM running a modern host kernel and OS as a guest this would also avoid issues with the host systems limitations. At which point I discovered the system had no virtualisation enabled apparently because the bootloader lacked support.

In addition to these software issues there were hardware problems, despite forcing the use of USB for all additional connectivity the USB implementation was dreadful. For a start all USB peripherals have to run at the same speed! One cannot mix full (12Mbit) and high speed (480Mbit) devices which makes adding a USB Ethernet and SATA device challenging when you cannot use a keyboard.

And because I needed more challenges only one of the USB root hubs was functional. In effect this made the console serial port critical as it was the only reliable way to reconfigure the system without a keyboard or network link (and WIFI was not reliable either)

After another long pause in proceedings I decided that I should house all the components together and that perhaps being out on my bench might be the cause of some instability. I purchased a powered Amazon basics USB 2 hub, an Ethernet adaptor and a USB 2 SATA interface in the hope of accessing some larger mass storage.

The USB hub power supply was 12V DC which matched the Hikey requirements so I worked out I could use a single 4A capable supply and run a 3.5inch SATA hard drive too. I designed a laser cut enclosure and mounted all the components. As it turned out I only had a 2.5inch hard drive handy so the enclosure is a little over size. If I were redoing this design I would attempt to make it fit in 1U of height and be mountable in a 19inch rack instead it is an 83mm high (under 2U) box.

A new software release had also become available which purported to use an UEFI bootloader after struggling to install this version unsuccessfully, made somewhat more problematic by the undocumented change from UART0 (unpopulated header) to UART3 on the low speed 2mm pitch header. The system seemed to start the kernel which panicked and hung either booting from eMMC or SD card. Once again the project went on hold after spending tens of hours trying to make progress.
Third time's a charmAs the year rolled to a close I once again was persuaded to look at the hikey, I followed the much improved instructions and installed the shiny new November software release which appears to have been made for the re-release of the Hikey through LeMaker. This time I obtained a Debian "jessie" system that booted from the eMMC.

Having a booted system meant I could finally try and use it. I had decided to try and use the system to host virtual machines used as builders within the NetSurf CI system.

The basic OS uses a mixture of normal Debian packages with some replacements from Linaro repositories. I would have prefered to see more use of Debain packages even if they were from the backports repositories but on the positive side it is good to see the use of Debian instead of Ubuntu.

The kernel is a heavily patched 3.18 built in a predominantly monolithic (without modules) manner with the usual exceptions such as the mali and wifi drivers (both of which appear to be binary blobs). The use of a non-mainline capable SoC means the standard generic distribution kernels cannot be used and unless Linaro choose to distribute a kernel with the feature built in it is necessary to compile your own from sources.

The default install has a linaro user which I renamed to my user and ensured all the ssh keys and passwords on the system were changed. This is an important step when using this pre-supplied images as often a booted system is identical to every other copy.

To access mass storage my only option was via USB, indeed to add any additional expansion that is the only choice. The first issue here is that the USB host support is compiled in so when the host ports are initialised it is not possible to select a speed other than 12MBit. The speed is changed to 480Mbit by using a user space application found in the users home directory (why this is not a tool provided by a package and held in sbin I do not know).

When the usb_speed tool is run there is a chance that the previously enumerated devices will be rescanned and what was /dev/sda has become /dev/sdb if this happens there is a high probability that the system must be rebooted to prevent random crashes due to the "zombie" device.

Because the speed change operation is unreliable it cannot be reliably placed in the boot sequence so this must be executed by hand on each boot to get access to the mass storage.

NetSurf project already uses a x86_64 virtual host system which runs an LLVM physical volume from which we allocate logical volumes for each VM. I initially hoped to do this with the hikey but as soon as I tried to use the logical volume with a VM the system locked up with nothing shown on console. I did not really try very hard to discover why and instead simply used files on disc for virtual drives which seemed to work.

To provide reliable network access I used a USB attached Ethernet device, this like the mass storage suffered from unreliable enumeration and for similar reasons could not be automated requiring manually using the serial console to start the system.

Once the system was started I needed to install the guest VM. I had hoped I might be able to install locally from Debian install media as I do for x86 using the libvirt tools. After a great deal of trial and error I finally was forced to abandon this approach when I discovered the Linaro kernel is lacking iso9660 support so installing from standard media was not possible.

Instead I used the instructions provided by Leif Lindholm to create a virtual machine image on my PC and copied the result across. These instructions are great except I used version 2.5 of Qemu instead of 2.2 which had no negative effect. I also installed the Debian backports for Jessie to get an up to date 4.3 kernel.

After copying the image to the Hikey I started it by hand from the command line as a four core virtual machine and was successfully able to log in. The guest would operate for up to a day before stopping with output such as

$
Message from syslogd@ciworker13 at Jan 29 07:45:28 ...
kernel:[68903.702501] BUG: soft lockup - CPU#0 stuck for 27s! [mv:24089]

Message from syslogd@ciworker13 at Jan 29 07:45:28 ...

kernel:[68976.958028] BUG: soft lockup - CPU#2 stuck for 74s! [swapper/2:0]

Message from syslogd@ciworker13 at Jan 29 07:47:39 ...
kernel:[69103.199724] BUG: soft lockup - CPU#3 stuck for 99s! [swapper/3:0]

Message from syslogd@ciworker13 at Jan 29 07:53:21 ...
kernel:[69140.321145] BUG: soft lockup - CPU#3 stuck for 30s! [rs:main Q:Reg:505]

Message from syslogd@ciworker13 at Jan 29 07:53:21 ...
kernel:[69192.880804] BUG: soft lockup - CPU#0 stuck for 21s! [jbd2/vda3-8:107]

Message from syslogd@ciworker13 at Jan 29 07:53:21 ...
kernel:[69444.805235] BUG: soft lockup - CPU#3 stuck for 22s! [swapper/3:0]

Message from syslogd@ciworker13 at Jan 29 07:55:21 ...
kernel:[69570.177600] BUG: soft lockup - CPU#1 stuck for 112s! [systemd:1]

Timeout, server 192.168.7.211 not responding.

After this output the host system would not respond and had to be power cycled never mind the guest!

Once I changed to single core operation the system would run for some time until the host suffered from the dreaded kernel OOM killer. I was at a loss as to why the oom killer was running as the VM was only allocated half the physical memory (512MB) allowing the host what I presumed to be an adequate amount.

By adding a 512MB swapfile the system was able to push the few hundred kilobytes it wanted to swap and the system was now stable! The swapfile of course has to be started by hand as the external storage is unreliable and unavailable at boot.

I converted the qemu command line to a libvirt config using the virsh tool
virsh domxml-from-native qemu-argv cmdln.args
The converted configuration required manual editing to get a working system but now I have a libvirt based VM guest I can control along with all my other VM using the virt-manager tool.

This system is now stable and has been in production use for a month at time of writing. The one guest VM is a single core 512MB aarch64 system which takes over 1100 seconds (19 minutes) to do what a Banana Pi 2 dual core 1GB memory 32bit native ARM system manages in 300 seconds.

It seems the single core limited memory system with USB SATA attached storage is very, very slow.

I briefly attempted to run the CI system job natively within the host system but within minutes it crashed hard and required a power cycle to retrieve, it had also broken the UEFI boot. I must thank Leif for walking me through recovering the system otherwise I would have needed to start over.
ConclusionsI must stress these conclusions and observations are my own and do not represent anyone else.

My main conclusions are:

This project has taken almost a year to get to the final state and has been one of the least enjoyable within that time. The only reason I have a running system at the end of it is sheer bloody mindedness because after spending hundreds of hours of my free time I was not prepared to see it all go to waste.

To be fair, the road I travelled is now much smoother and if the application is suited to having a mobile phone without a screen then the Hikey probably works as a solution. For me, however, the Hikey product with the current hardware and software limitations is not something I would recommend in preference to other options.

Erich Schubert: Stop abusing lambda expressions - this is not functional programming

I know, all the Scala fanboys are going to hate me now. But:
Stop overusing lambda expressions.
Most of the time when you are using lambdas, you are not even doing functional programming, because you often are violating one key rule of functional programming: no side effects.
For example:
collection.forEach(System.out::println);
is of course very cute to use, and is (wow) 10 characters shorter than:
for (Object o : collection) System.out.println(o);
but this is not functional programming because it has side effects.
What you are doing are anonymous methods/objects, using a shorthand notion. It's sometimes convenient, it is usually short, and unfortunately often unreadable, once you start cramming complex problems into this framework.
It does not offer efficiency improvements, unless you have the propery of side-effect freeness (and a language compiler that can exploit this, or parallelism that can then call the function concurrently in arbitrary order and still yield the same result).
Here is an examples of how to not use lambdas:
DZone Java 8 Factorial (with boilerplate such as the Pair class omitted):
Stream<Pair> allFactorials = Stream.iterate(
  new Pair(BigInteger.ONE, BigInteger.ONE),
  x -> new Pair(
    x.num.add(BigInteger.ONE),
    x.value.multiply(x.num.add(BigInteger.ONE))));
return allFactorials.filter(
  (x) -> x.num.equals(num)).findAny().get().value;
When you are fresh out of the functional programming class, this may seem like a good idea to you... (and in contrast to the examples mentioned above, this is really a functional program).
But such code is a pain to read, and will not scale well either. Rewriting this to classic Java yields:
BigInteger cur = BigInteger.ONE, acc = BigInteger.ONE;
while(cur.compareTo(num) <= 0)  
  cur = cur.add(BigInteger.ONE); // Unfortunately, BigInteger is immutable!
  acc = acc.multiply(cur);
 
return acc;
Sorry, but the traditional loop is much more readable. It will still not perform very well (because of BigInteger not being designed for efficiency - it does not even make sense to allow BigInteger for num - the factorial of 2**63-1, the maximum of a Java long, needs 1020 bytes to store, i.e. about 500 exabyte.
For some, I did some benchmarking. One hundred random values num (of course the same for all methods) from the range 1 to 1000.
I also included this even more traditional version:
BigInteger acc = BigInteger.ONE;
for(long i = 2; i <=x; i++)  
  acc = acc.multiply(BigInteger.valueOf(i));
 
return acc;
Here are the results (Microbenchmark, using JMH, 10 warum iterations, 20 measurement iterations of 1 second each):
functional    1000     100  avgt   20  9748276,035   222981,283  ns/op
biginteger    1000     100  avgt   20  7920254,491   247454,534  ns/op
traditional   1000     100  avgt   20  6360620,309   135236,735  ns/op
As you can see, this "functional" approach above is about 50% slower than the classic for-loop. This will be mostly due to the Pair and additional BigInteger objects created and garbage collected.
Apart from being substantially faster, the iterative approach is also much simpler to follow. (To some extend it is faster because it is also easier for the compiler!)
There was a recent blog post by Robert Br utigam that discussed exception throwing in Java lambdas and the pitfalls associated with this. The discussed approach involves abusing generics for throwing unknown checked exceptions in the lambdas, ouch.

Don't get me wrong. There are cases where the use of lambdas is perfectly reasonable. There are also cases where it adheres to the "functional programming" principle. For example, a stream.filter(x -> x.name.equals("John Doe")) can be a readable shorthand when selecting or preprocessing data. If it is really functional (side-effect free), then it can safely be run in parallel and give you some speedup.
Also, Java lambdas were carefully designed, and the hotspot VM tries hard to optimize them. That is why Java lambdas are not closures - that would be much less performant. Also, the stack traces of Java lambdas remain somewhat readable (although still much worse than those of traditional code). This blog post by Takipi showcases how bad the stacktraces become (in the Java example, the stream function is more to blame than the actual lambda - nevertheless, the actual lambda application shows up as the cryptic LmbdaMain$$Lambda$1/821270929.apply(Unknown Source) without line number information). Java 8 added new bytecodes to be able to optimize Lambdas better - earlier JVM-based languages may not yet make good use of this.
But you really should use lambdas only for one-liners. If it is a more complex method, you should give it a name to encourage reuse and improve debugging.
Beware of the cost of .boxed() streams!
And do not overuse lambdas. Most often, non-Lambda code is just as compact, and much more readable. Similar to foreach-loops, you do lose some flexibility compared to the "raw" APIs such as Iterators:
for(Iterator<Something>> it = collection.iterator(); it.hasNext(); )  
  Something s = it.next();
  if (someTest(s)) continue; // Skip
  if (otherTest(s)) it.remove(); // Remove
  if (thirdTest(s)) process(s); // Call-out to a complex function
  if (fourthTest(s)) break; // Stop early
 
In many cases, this code is preferrable to the lambda hacks we see pop up everywhere these days. Above code is efficient, and readable.
If you can solve it with a for loop, use a for loop!
Code quality is not measured by how much functionality you can do without typing a semicolon or a newline!
On the contrary: the key ingredient to writing high-performance code is the memory layout (usually) - something you need to do low-level.
Instead of going crazy about Lambdas, I'm more looking forward to real value types (similar to a struct in C, reference-free objects) maybe in Java 9 (Project Valhalla), as they will allow reducing the memory impact for many scenarios considerably. I'd prefer a mutable design, however - I understand why this is proposed, but the uses cases I have in mind become much less elegant when having to overwrite instead of modify all the time.

29 February 2016

Tom Marble: is-slfc-shooting-open-source-in-the-foot

Is SFLC Shooting Open Source in the Foot? The academic article by SFLC about ZFS is troubling and may unintentionally shoot free software licensing in the foot. When I was at Sun (as part of the team that released the Java Programming Language by starting the OpenJDK project) I often heard community concerns about the CDDL license. At the time the big complaint was about the "Choice of Venue" clause. I got involved because Sun had developed many essential Java libraries and distributed them under CDDL. The community requested a more permissive license and I was able to convince internal project leaders (and Sun's lawyers) to make a licensing change for a handful of these projects. And there was much rejoicing. Based on my experience in helping Java to become open source I came to appreciate the legal hacks on copyright which make open source possible. It's the free software license which uses copyright to enable sharing (vs. the default of disabling sharing).

Open Source Licenses
And so I have appreciated many of the writings and speeches from SFLC on the mechanisms of software freedom. I was particularly moved by the talks about the "Freedom Box" concept. That's why this SFLC post on ZFS sounds so off key: if open source works because of free software licenses it seems weird to weaken that foundation by prioritizing the "equity" (or intended spirit) of the license. Allow me to mention that as I do most of my computing these days on GNU/Linux I miss the super cool features of ZFS from Solaris. I did try an early version of btrfs and was quite disappointed (but that's another story). In this happy case the source code for ZFS is available, but what about the future, when we aren't so lucky and someone asserts in court that the "you know, the software license was really about the spirit of sharing and that means we are allowed to use it -- and not be held to the pesky details as written in the license". A lawyer I respect called this out: "Equity" has no place in US law. The point is that for lawyers software licenses work because they have clear, written rules to guarantee the spirit is upheld; but spirit doesn't work in front of a judge -- clear rules do. Free and open source software has made so much progress in all facets of life why on earth would we second guess the licensing tools that made it possible? And why would SFLC try to shift the spotlight (and in this case the legal burden) to "a good-faith belief that the conduct falls within the equity of the license". Especially given the earlier comment which clearly states "[the combination] is inconsistent with the literal meaning of GPLv2 section 2(b)."
Wat?

The entire raison d' tre for open source software licenses was so that developers (and users) would have clarity and wouldn't have to ask permission to use the software!!! As stated elsewhere (and like I did with those Java libraries) the easy solution is to have the ZFS copyright holder (now Oracle) reclicense (or dual license) the code under a compatible license (permissive or copyleft). If OpenSolaris was still a thing I might understand some hesitancy, but why not liberate ZFS now? So we have to wonder what could possibly be motivating this odd "spirit of the license" position on the part of SFLC? Fortunately charities that enjoy non-profit status are required to make public filings of their income in something called a "Form 990". The latest SFLC 990 I could find shows SFLC getting 78% (or just over $5 million) from "non public support" (see page 14). A number with "two commas" would even be interesting to for-profit companies. Just whom is making these "donations" and what exactly do they get in return? Apparently I'm not the only one wondering about this question. On one hand it's important to know if SFLC as a non-profit is, indeed, acting in the public interest (as the IRS requires). Yet the even bigger issue here is would "asking for a consensus about the spirit" trump the written copyright license and set a scary precedent for open source software in general?

Daniel Silverstone: Kicad hacking - Intra-sheet links and ERC

This is a bit of an odd posting since it's about something I've done but is also here to help me explain why I did it and thus perhaps encourage some discussion around the topic within the Kicad community...
Recently (as you will know if you follow this blog anywhere it is syndicated) I have started playing with Kicad for the development of some hardware projects I've had a desire for. In addition, some of you may be aware that I used to work for a hardware/software consultancy called Simtec, and there I got to play for a while with an EDA tool called Mentor Designview. Mentor was an expensive, slow, clunky, old-school EDA tool, but I grew to understand and like the workflow. I spent time looking at gEDA and Eagle when I wanted to get back into hardware hacking for my own ends; but neither did I really click with. On the other hand, a mere 10 minutes with Kicad and I knew I had found the tool I wanted to work with long-term. I designed the beer'o'meter project (a flow meter for the pub we are somehow intimately involved with) and then started on my first personal surface-mount project -- SamDAC which is a DAC designed to work with our HiFi in our study at home. As I worked on the SamDAC project, I realised that I was missing a very particular thing from Mentor, something which I had low-level been annoyed by while looking at other EDA tools -- Kicad lacks a mechanism to mark a wire as being linked to somewhere else on the same sheet. Almost all of the EDA tools I've looked at seem to lack this nicety, and honestly I miss it greatly, so I figured it was time to see if I could successfully hack on Kicad. Kicad is written in C++, and it has been mumble mumble years since I last did any C++, either for personal hacking or professionally, so it took a little while for that part of my brain to kick back in enough for me to grok the codebase. Kicad is not a small project, taking around ten minutes to build on my not-inconsiderable computer. And while it beavered away building, I spent time looking around the source code, particularly the schematic editor eeschema. To skip ahead a bit, after a couple of days of hacking around, I had a proof-of-concept for the intra-sheet links which I had been missing from my days with Mentor, and some ERC (electrical rules checking) to go alongside that to help produce schematics without unwanted "sharp corners". In total, I added:
  • an "intra-sheet link" schematic element, including: Intra-sheet linking tool
    • the schematic element itself
    • loading and saving of the element (which required a bump in schematic file format versioning)
    • UI for adding intra-sheet links to your schematic
    • netlist build support to ensure that the links found their way into the internal netlist content.
  • an ERC tickybox "Strict wiring checks" and behind that I hid: Strict wiring checks
    • checks for wires with dangling ends (dangles are bad)
    • junctions which have no point in being present (pointless junctions are confusing to readers)
    • checks that intra-sheet links exist in groups of two or more on any given net, since individual links are pointless.
I forked the Kicad mirror on Github and pushed my own branch with this work to my Kicad fork. All of this is meant to allow schematic capture engineers to more clearly state their intentions regarding what they are drawing. The intra-sheet link could be thought of like a no-connect element, except instead of saying "this explicitly goes nowhere" we're saying "this explicitly goes somewhere else on this sheet, you can go look for it". Obviously, people who dislike (or simply don't want to use) such intra-sheet link elements can just disable that ERC tickybox and not be bothered by them in the least (well except for the toolbar button and menu item I suppose). Whether this work gets accepted into Kicad, or festers and dies on the vine, it was good fun developing it and I'd like to illustrate how it could help you, and why I wrote it in the first place:

A contrived story
Note, while this story is meant to be taken seriously, it is somewhat contrived, the examples are likely electrical madness, but please just think about the purpose of the checks etc.
To help to illustrate the feature and why it exists, I'd like to tell you a somewhat contrived story about Fred. Fred is a schematic capture engineer and his main job is to review schematics generated by his colleagues. Fred and his colleagues work with Kicad (hurrah) but of late they've been having a few issues with being able to cleanly review schematics. Fred's colleagues are not the neatest of engineers. In particular they tend to be quite lazy when it comes to running busses, which are not (for example) address and data busses, around their designs and they tend to simply have wires which end in mid-space and pick up somewhere else on the sheet. All this is perfectly reasonable of course, and Kicad handles it with aplomb. Sadly it seems quite error prone for Fred's workplace. As an example, Fred's colleague Ben has been designing the power supply for a particular board. As with most power supplies, plenty of capacitors are needed to stabilise the regulators and smooth the output. In the example below, the intent is that all of the capacitors are on the FOO net. Contrived problem example 1 Sadly there's a missing junction and/or slightly misplaced label in the upper section which means that C2 and C3 simply don't join to the FOO net. This could easily be missed, but the ERC can't spot it at all since there's more than one thing on each net, so the pins of the capacitors are connected to something. Fred is very sad that this kind of problem can sometimes escape notice by the schematic designer Ben, Fred himself, and the layout engineer, resulting in boards which simply do not work. Fred takes it upon himself to request that the strict wiring checks ERC is made mandatory for all designs, and that the design engineers be required to use intra-sheet link symbols when they have signals which wander off to other parts of the sheet like FOO does in the example. Without any further schematic changes, strict wiring checks enabled gives the following points of ERC concern for Ben to think about: Contrived problem example 2 As you can see, the ERC is pointing at the wire ends and the warnings are simply that the wires are dangling and that this is not acceptable. This warning is very like the pin-not-connected warnings which can be silenced with an explicit no-connect schematic element. Ben, being a well behaved and gentle soul, obeys the design edicts from Fred and seeks out the intra-sheet link symbols, clearing off the ERC markers and then adding intra-sheet links to his design: Contrived problem example 3 This silences the dangling end ERC check, which is good, however it results in another ERC warning: Contrived problem example 4 This time, the warning for Ben to deal with is that the intra-sheet links are pointless. Each exists without a companion to link to because of the net name hiccough in the top section. It takes Ben a moment to realise that the mistake which has been made is that a junction is missing in the top section. He adds the junction and bingo the ERC is clean once more: Contrived problem example 5 Now, this might not seem like much gain for so much effort, but Ben can now be more confident that the FOO net is properly linked across his design and Fred can know, when he looks at the top part of the design, that Ben intended for the FOO net to go somewhere else on the sheet and he can look for it.

Why do this at all? Okay, dropping out of our story now, let's discuss why these ERC checks are worthwhile and why the intra-sheet link schematic element is needed.
Note: This bit is here to remind me of why I did the work, and to hopefully explain a little more about why I think it's worth adding to Kicad...
Designers are (one assumes) human beings. As humans we (and I count myself here too) are prone to mistakes. Sadly mistakes are often subtle and could easily be thought of as deliberate if the right thought processes are not followed carefully when reviewing. Anyone who has ever done code review, proofread a document, or performed any such activity, will be quite familiar with the problems which can be introduced by a syntactically and semantically valid construct which simply turns out to be wrong in the greater context. When drawing designs, I often end up with bits of wire sticking out of schematic sections which are not yet complete. Sadly if I sleep between design sessions, I often lose track of whether such a dangling wire is meant to be attached to more stuff, or is simply left because the net is picked up elsewhere on the sheet. With intra-sheet link elements available, if I had intended the latter, I'd have just dropped such an element on the end of the wire before I stopped for the day. Also, when drawing designs, I sometimes forget to label a wire, especially if it has just passed through a filter or current-limiting resistor or similar. As such, even with intra-sheet link elements to show me when I mean for a net to go bimbling off across the sheet, I can sometimes end up with unnamed nets whose capacitors end up not used for anything useful. This is where the ERC comes in. By having the ERC complain if a wire dangles -- the design engineer won't forget to add links (or check more explicitly if the wire is meant to be attached to something else). By having junctions which don't actually link anything warned about, the engineer can't just slap a junction blob down on the end of a wire to silence that warning, since that doesn't mean anything to a reviewer later down the line. By having the ERC warn if a net has exactly one intra-sheet link attached to it, missing net names and errors such as that shown in my contrived example above can be spotted and corrected. Ultimately this entire piece of work is about ensuring that the intent of the design engineer is captured clearly in the schematic. If the design engineer meant to leave that wire dangling because it's joining to another bit of wire elsewhere on the sheet, they can put the intra-sheet links in to show this. The associated ERC checks are there purely to ensure that the validation of this intent is not bypassed accidentally, or deliberately, in order to make the use of this more worthwhile and to increase the usefulness of the ERC on designs where signals jump around on sheets where wiring them up directly would just create a mess.

25 February 2016

Russell Coker: Ethernet Interface Naming With Systemd

Systemd has a new way of specifying names for Ethernet interfaces as documented in systemd.link(5). The Debian package should keep working with the old 70-persistent-net.rules file, but I had a problem with this that forced me to learn about systemd.link(5). Below is a little shell script I wrote to convert a basic 70-persistent-net.rules (that only matches on MAC address) to systemd.link files. #!/bin/bash RULES=/etc/udev/rules.d/70-persistent-net.rules for n in $(grep ^SUB $RULES sed -e s/^.*NAME..// -e s/.$//) ; do
NAME=/etc/systemd/network/10-$n.link
LINE=$(grep $n $RULES)
MAC=$(echo $LINE sed -e s/^.*address .// -e s/ ATTR.*$//)
echo "[Match]" > $NAME
echo "MACAddress=$MAC" >> $NAME
echo "[Link]" >> $NAME
echo "Name=$n" >> $NAME
done

22 February 2016

Norbert Preining: The Good, the Bad, and the Ugly refugees acceptance

Coming from a country that has taken lots of refugees, especially in the years after the second world war (although not any more), and while living in Japan still feeling connected to Europe and what is going on there, the current situation of refugees from the near east gives lots of material to think about. One of these things recently was passed around on various SNS sites. It is about the number of refugees a country can accept . refugees The situation is difficult, and after an initial wave of empathy it seems that especially governments are retracting and trying to stop the influx of refugees:
We cannot accommodate any more refugees in Europe, that s not possible.
Manuel Valls, Prime Minister of France
I am well aware about the difficult situation in Europe, and consider the dealing with the refugee crisis the biggest challenge of these decade. In this blog, without many words, I only want to focus on how Japan deals with refugees. Europe might be bad, but Japan I am bare of words.
Lebanon European Community Japan
Population 4.467.000 (2013) 503.000.000 127.300.000 (2013)
Refugees 2015 ~1.000.000 850.000 27
Ratio 4.4 : 1 590 : 1 4714814 : 1
Sources of the data: No words are necessary.

12 February 2016

Benjamin Mako Hill: Unhappy Birthday Suspended

More than 10 years ago, I launched Unhappy Birthday in a fit of copyrighteous exuberance. In the last decade, I have been interviewed on the CBC show WireTap and have received an unrelenting stream of hate mail from random strangers. With a recently announced settlement suggesting that Happy Birthday is on its way into the public domain, it s not possible for even the highest-protectionist in me to justify the continuation of the campaign in its original form. As a result, I ve suspended the campaign while I plan my next move. Here s the full text of the notice I posted on the Unhappy Birthday website:
Unfortunately, a series of recent legal rulings have forced us to suspend our campaign. In 2015, Time Warner s copyright claim to Happy Birthday was declared invalid. In 2016, a settlement was announced that calls for a judge to officially declare that the song is in the public domain. This is horrible news for the future of music. It is horrible news for anybody who cares that creators, their heirs, etc., are fairly remunerated when their work is performed. What incentive will there be for anybody to pen the next Happy Birthday knowing that less than a century after their deaths their estates and the large multinational companies that buy their estates might not be able to reap the financial rewards from their hard work and creativity? We are currently planning a campaign to push for a retroactive extension of copyright law to place Happy Birthday, and other works, back into the private domain where they belong! We believe this is a winnable fight. After all, copyright has been retroactively extended before! Stay tuned! In the meantime, we ll keep this page here for historical purposes.

Copyrighteous Benjamin Mako Hill (2016-02-11)

8 February 2016

Joachim Breitner: Protecting static content with mod_rewrite

Since fourteen years, I have been photographing digitally and putting the pictures on my webpage. Back then, online privacy was not a big deal, but things have changed, and I had to at least mildly protect the innocent. In particular, I wanted to prevent search engines from accessing some of my pictures. As I did not want my friends and family having to create an account and remember a password, I set up an OpenID based scheme five years ago. This way, they could use any of their OpenID enabled account, e.g. their Google Mail account, to log in, without disclosing any data to me. As my photo album consists of just static files, I created two copies on the server: the real one with everything, and a bunch of symbolic links representing the publicly visible parts. I then used mod_auth_openid to prevent access to the protected files, unless the users logged in. I never got around of actually limiting who could log in, so strangers were still able to see all photos, but at least search engine spiders were locked out. But, very unfortunately, OpenID did never really catch on, Google even stopped being a provider, and other promising decentralized authentication schemes like Mozilla Persona are also phased out. So I needed an alternative. A very simply scheme would be a single password that my friends and family can get from me and that unlocks the pictures. I could have done that using HTTP Auth, but that is not very user-friendly, and the login does not persist (at least not without the help of the browser). Instead, I wanted something that involves a simple HTTP form. But I also wanted to avoid server-side programming, for performance and security reasons. I love serving static files whenever it is feasible. Then I found that mod_rewrite, Apache s all-around-tool for URL rewriting and request mangling, supports reading and writing cookies! So I came up with a scheme that implements the whole login logic in the Apache server configuration. I d like to describe this setup here, in case someone finds it inspiring. I created a login.html with a simple HTML form:
<form method="GET" action="/bilder/login.html">
 <div style="text-align:center">
  <input name="password" placeholder="Password" />
  <button type="submit">Sign-In</button>
 </div>
</form>
It sends the user to the same page again, putting the password into the query string, hence the method="GET" mod_rewrite unfortunately cannot read the parameters of a POST request. The Apache configuration is as follows:
RewriteMap public "dbm:/var/www/joachim-breitner.de/bilder/publicfiles.dbm"
<Directory /var/www/joachim-breitner.de/bilder>
 RewriteEngine On
 # This is a GET request, trying to set a password.
 RewriteCond % QUERY_STRING  password=correcthorsebatterystaple
 RewriteRule ^login.html /bilder/loggedin.html [L,R,QSD,CO=bilderhp:correcthorsebatterystaple:www.joachim-breitner.de:2000000:/bilder]
 # This is a GET request, trying to set a wrong password.
 RewriteCond % QUERY_STRING  password=
 RewriteRule ^login.html /bilder/notloggedin.html [L,R,QSD]
 # No point in loggin in if there is already the right password
 RewriteCond % HTTP:Cookie  bilderhp=correcthorsebatterystaple
 RewriteRule ^login.html /bilder/loggedin.html [L,R]
 # If protected file is requested, check for cookie.
 # If no cookie present, redirect pictures to replacement picture
 RewriteCond % HTTP:Cookie  !bilderhp=correcthorsebatterystaple
 RewriteCond $ public:$0 private  private
 RewriteRule ^.*\.(png jpg)$ /bilder/pleaselogin.png [L]
 RewriteCond % HTTP:Cookie  !bilderhp=correcthorsebatterystaple
 RewriteCond $ public:$0 private  private
 RewriteRule ^.+$ /bilder/login.html [L,R]
</Directory>
The publicfiles.dbm file is generated from a text file with lines like
login.html.en 1
login.html.de 1
pleaselogin.png 1
thumbs/20030920165701_thumb.jpg 1
thumbs/20080813225123_thumb.jpg 1
...
using
/usr/sbin/httxt2dbm -i publicfiles.txt -o publicfiles.dbm
and whitelists all files that are visible without login. Make sure it contains the login page, otherwise you ll get a redirect circle. The other directives in the above configure fulfill these tasks:
  • If the password (correcthorsebatterystaple) is in the query string, the server redirects the user to a logged-in-page that tells him that the login was successful and instructs him to reload the photo album. It also sets a cookie that will last very long -- after all, I want this to be convenient for my visitors. The query string parsing is not very strict (e.g. a password of correcthorsebatterystaplexkcdrules would also work), but that s ok.
  • The next request detects an attempt to set a password. It must be wrong (otherwise the first rule would have matched), so we redirect the user to a variant of the login page that tells him so.
  • If the user tries to access the login page with a valid cookie, just log him in.
  • The next two rules implement the actual protection. If there no valid cookie and the accessed file is not whitelisted, then access is forbidden. For requests to images, we do an internal redirect to a placeholder image, while for everything else we redirect the user to the login page.
And that s it! No resource-hogging web frameworks, not security-dubious scripting languages, and a dead-simple way to authenticate. Oh, and if you believe you know me well enough to be allowed to see all photos: The real password is not correcthorsebatterystaple; just ask me what it is.

7 February 2016

Mike Hommey: SSH through jump hosts, revisited

Close to 7 years ago, I wrote about SSH through jump hosts. Twice. While the method used back then still works, Openssh has grown an new option in version 5.3 that allows it to be simplified a bit, by not using nc. So here is an updated rule, version 2016:
Host *+*
ProxyCommand ssh -W $(echo %h   sed 's/^.*+//;s/^\([^:]*$\)/\1:22/') $(echo %h   sed 's/+[^+]*$//;s/\([^+%%]*\)%%\([^+]*\)$/\2 -l \1/;s/:\([^:+]*\)$/ -p \1/')
The syntax you can use to connect through jump hosts hasn t changed compared to previous blog posts:
  • With one jump host:
    $ ssh login1%host1:port1+host2:port2 -l login2
  • With two jump hosts:
    $ ssh login1%host1:port1+login2%host2:port2+host3:port3 -l login3
  • With three jump hosts:
    $ ssh login1%host1:port1+login2%host2:port2+login3%host3:port3+host4:port4 -l login4
  • etc.
Logins and ports can be omitted. Update: Add missing port to -W flag when one is not given.

31 January 2016

Martin-&#201;ric Racine: Stupid Design: Facebook's Selling Something feature

Over these past few months, I've been going through my personal belongings to reduce them to the bare essentials more-or-less following the rule "If I cannot remember that I had it, I probably don't need it" with an explicit goal to make it easier for me to relocate in the near future.Part of that iterative process has involved selling surplus items via Facebook groups. While I initially rejoiced at the invention of the new Selling Something feature, the more I use it, the more I end up cursing at the lack of thought that went into designing that feature:For instance, in the feature's most recent implementation, it has become possible to post the same ad in several groups. Great idea, right? Actually, no. What this option does is post individual copies of that exact same ad in each group. Managed to sell a particular item? You'll have to sort through each and every duplicate post in each group to mark the item as sold. Need to edit the ad to lower the price or add requested information about the item? Same thing: go through each individual copy of the ad in each group. Seriously, Facebook? Who the fuck designed that senseless implementation?!Here's a better implementation: centrally manage all items via the user's account preferences, and let the user tick checkboxes besides each item to decide which groups will see that particular item. Ditto whenever editing the ad's content or marking an item as sold; do everything via the user's account preferences. Rather than making the content group-centric, make it user-centric.You're welcome.

30 January 2016

Lunar: Only allow outgoing connections through Tor

Using Tor helps take back control of what data one decides to share online. It also helps in avoiding surveillance from the local network (think of someone listening to the public Wi-Fi) or the Internet access provider. But it is easy to make mistakes: Tor Browser, the easiest and most common way to use Tor, is just a web browser. Extra special care is required to make other applications use Tor, and it is easy to forget to change settings, let alone actual attackers. That's why using Tails is often recommended as everything is preconfigured and it requires traffic to go through Tor by various means. Qubes or Whonix also have interesting properties, but require reinstalling systems and changing habits. Meanwhile, securing the network on an already installed system can help you feel better. A potential solution to prevent connections from leaking out without going through Tor is simply to prevent all connections from leaving the computer, except the ones going to the Tor network. This can quite easily be implemented by setting up an outgoing firewall. Using the default settings, a Tor client can potentially connect to any of the 2,000 Guard relays. While there are ways to get such a list and turn it into firewall rules, we are going to use an easier solution. Because the list of Tor relays is public, censors have often used it to prevent people from connecting to the Tor network. The Tor project came up with bridges as a work-around. Bridges are entry nodes in the Tor network. For public bridges, users can only get a few bridge addresses at a time, preventing an adversary from easily blocking them at all once. When configured to use a set of bridges, Tor will only connect to their addresses. This will make our firewall much shorter. The instructions that follow should work on Debian Jessie and later versions. Adapting them to other systems should not be too hard either. We assume that Tor Browser is already installed. As we will want other applications to use Tor than just the browser, we also want to install a system-wide instance:
# apt install tor
Next, visit bridges.torproject.org to request a set of bridges: BridgeDB landing page You then need to select the type of bridges. Tor standard protocol is easy to identify as such. The Tor project has developed pluggable transports to easily disguise the traffic, but using them is outside the scope for this documentation. So lets stay with the default type of bridges: BridgeDB bridge type selection After solving a CAPTCHA, you'll get a list of addresses and fingerprints for a couple of bridges like the following one: Example bridge list from BridgeDB (blurred to protect the actual bridges To tell Tor to use these bridges, edit /etc/tor/torrc, and at the end, add something like:
UseBridges 1
Bridge 109.XXX.XXX.XXX:4XXX 2244XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Bridge 37.XXX.XXX.XXX:1XXXX 9CF0XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Bridge 171.XXX.XXX.XXX:4XXX 7705XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Replace the actual addresses and fingerprints by those you got earlier. Now reload the daemon and make sure that it still connects to the network:
# service tor reload
# tail -f /var/log/tor/log
Tor Browser also needs to be told to use these bridges. Open the onion menu close to the location bar, and select Tor Network Settings . Tor Browser  onion menu In there, tick My Internet Service Provider (ISP) blocks connections to the Tor network and paste the list of bridges below Enter custom bridges: Tor Browser Network Settings Before we restrict outgoing connections to only use the Tor network, lets configure APT to use Tor as well so we can continue installing and upgrading packages in the future:
# apt install apt-transport-tor
The following command will reconfigure APT to use Tor to access the currently configured package repositories. It basically takes care of replacing every http:// by tor+http://:
# sed -e 's/^[^#]*\(deb\(-src\)\?\W\+\)http/\1tor+http/' -i /etc/apt/sources.list /etc/apt/sources.list.d/*.list
Lets see if it still works:
# apt update
One last thing: sometimes, we still need to bypass our restrictions and access the network directly. Some websites sadly deny access to Tor users, or we might need to login through a captive portal before being able to use a random network. To make it practical, we are going to create a new user account. This will enable us to use the desktop user switching capabilities to start a new browser or sudo for command line applications. In this example, the user is going to be named passthrough:
# adduser passthrough
Make sure to set a proper password! Now lets configure the firewall. Here we are going to use ferm which makes it fairly easy to do low-level firewall rule in a readable manner:
# apt install ferm
We want the firewall to be started at boot time. Now edit its configuration in /etc/ferm/ferm.conf. At the end, add:
# Let's block everything for both IPv4 and IPv6, except to localhost,
# already established connections and the special 'passthrough' user.
domain (ip ip6) chain OUTPUT  
    policy DROP;
    outerface lo ACCEPT;
    proto icmp ACCEPT;
    mod state state (ESTABLISHED RELATED) ACCEPT;
    mod owner uid-owner "passthrough" ACCEPT;
 
# Now for IPv4, we want to restrict output to configured Tor bridges
domain ip chain OUTPUT  
    # Don't break network autoconfiguration
    mod owner uid-owner 0 proto udp dport bootps ACCEPT;
    # Allow bridges defined in torrc by getting them directly
    # from the configuration file.
    @include "sed -n -e 's/^Bridge \([^ ]* \)\?\([0-9.]*\):\([0-9]*\).*/daddr \2 proto tcp dport \3 ACCEPT;/p' /etc/tor/torrc  ";
 
# Let's REJECT everything else so we get notice instead of timeouts
domain (ip ip6) chain OUTPUT REJECT;
Reload the firewall through:
# service ferm reload
Lets do some tests:
# ping check.torproject.org
  ping: unknown host check.torproject.org
# tor-resolve check.torproject.org
  38.229.72.22
# ping 38.229.72.22
  From 192.168.7.234 icmp_seq=1 Destination Port Unreachable
# curl https://check.torproject.org/
  curl: (6) Could not resolve host: check.torproject.org
# curl -I https://38.229.72.22/
  curl: (7) Failed to connect to 38.229.72.22 port 443: Connection refused
# curl --socks5 127.0.0.1:9050 https://check.torproject.org/
  curl: (6) Could not resolve host: check.torproject.org
# curl --silent --socks5-hostname 127.0.0.1:9050 https://check.torproject.org/   grep Congratulations
  Congratulations. This browser is configured to use Tor.
# sudo -u passthrough ping -c 1 check.torproject.org
  64 bytes from sergii.torproject.org (38.229.72.22): icmp_seq=1 ttl=54 time=148 ms
# sudo -u passthrough curl --silent https://check.torproject.org/   grep Sorry
  Sorry. You are not using Tor.
One last thing to complete the setup: for command line applications like SSH, torsocks does wonders. Just prefix your usual commands by torsocks and they should work:
# apt install torsocks
# torsocks curl --silent https://check.torproject.org/   grep Congratulations
  Congratulations. This browser is configured to use Tor.
All set! In case you ever need to turn it all off, just ask ferm to reconfigure the firewall:
# service ferm stop
But don't forget to turn it back on later!
# service ferm start
Ideally, we would have a way to automatically configure a different set of bridges for each network we connect to make it more difficult to trace the computer from network to network. But there's quite some more work to be done on various levels before that can be done in an integrated manner (torshiftchange and tordyguards are promising options). Meanwhile, be aware that you might want to keep a fresh set of bridges handy to configure them after leaving for a trip. Another downside of the setup currently is that it doesn't allow for network time synchronization. That might make the system fingerprintable by using its clock drift. To use obfs4 bridges which helps defeating censored networks, you will need two extra steps. First, install the required software:
# apt install obfs4proxy
Then add the following extra line to /etc/tor/torrc:
ClientTransportPlugin obfs2,obfs3,obfs4,scramblesuit exec /usr/bin/obfs4proxy
You can then request obfs4 bridges from BridgeDB. These changes are only required for the system-wide Tor daemon, as Tor Browser comes with obs4proxy preconfigured. Kudos to Micah for proofreading this article.

28 January 2016

Joachim Breitner: Dreaming of role playing

Recently, at a summer-school-like event, we were discussing pen-and-paper role playing. I m not sure if this was after a session of role-playing, but I was making the point that you don t need much or any at all of the rules, and scores, and dice, if you are one of the story-telling role players, and it can actually be more fun this way. As an example, I said, it can make sense if one of the players (and the game master, I suppose) reads up a lot about one aspect of the fantasy world, e.g. one geographical area, one cult, one person, and then this knowledge is used to create an exciting puzzle, even without any opponents. I m not quite sure, but I think I fell asleep shortly after, and I dreamed of such a role playing session. It was going roughly like this:
I (a human), and my fellows (at least a dwarf, not sure about the rest) went to some castle. It was empty, but scary. We crossed its hall, and went into a room on the other side. It was locked towards the hall by a door that covered the door frame only partly, and suddenly we could see a large Ogre, together with other foul folk not worth mentioning, hammered at the door. My group (which was a bit larger in that moment) all prepared shooting arrows at him the moment it burst through the door. I had the time to appreciate the ingenuity that we all waited for him to burst through, so that none of the arrows would bounce of the door, but it did not help, and we ran from the castle, over a field, through a forest, at the other side of which we could see, below a sleep slope, a house, so we went there. The path towards that was filled with tracks that looked surprisingly like car tracks. When we reached the spot there was no house any more, but rather a cold camp side. We saw digging tools, and helmets (strangely, baseball helmets) were arranged in a circle, as if it was a burial site. We set up camp there and slept. It occurred to me that I must have been the rightful owner of the castle, and it was taken by me from my brother and his wife, who denied my existence or something treacherously like that. When we woke up at the camp side, she were there, together with what must be my niece. My sister in law mocked us for fighting unsuccessfully at the castle, but my niece was surprised to see me, as I must have a very similar appearance to my brother. She said that her mother forbid it, but she nevertheless sneakily takes out something which looks like a Gameboy with a camera attachment and a CompactFlash card from her mothers purse, puts it in and take a photo of me. This is when I realize that I will get my castle back.
At that moment, I woke up. I somewhat liked the story (and it was a bit more coherent in my mind than what I then wrote down here), so I wanted to write it down. I quickly fetched my laptop. My friends at the summer school were a bit worried, and I promised not to mention their names and concrete places, and started writing. They distracted me, so I searched for a place of my own, lied down (why? no idea), and continued writing. I had to to touch writing on my belly, because my laptop was not actually there. I also noticed that I am back at the camp side, and that I am still wearing my back protector that I must have been wearing while fighting in the castle, and which I did not take off while sleeping at the camp side. Funnily, it was not a proper medieval amour, but rather my snowboarding back protector.
At that moment, I woke up. I somewhat liked the story (and it was a bit more coherent in my mind than what I then wrote down here), so I wanted to write it down. I quickly got up, started my laptop, and wrote it down. And this is what you are reading right now. Off to bed again, let s see what happens next.

27 January 2016

Lunar: ASLR now!

Address space layout randomization helps to protect against buffer overflow attacks as it becomes harder for an attacker to turn a programming error into an exploitable security hole. The first implementation for Linux is 15 years old. Support in mainline kernel and toolchains have been available for a good while now. But to work, ASLR also needs executables to be built as position independent. And as Hanno B ck from the fuzzing project gently reminded me at 32C3, almost no executables in Debian are built as such, while it is now the default in Windows, Mac OS X, OpenBSD, and Fedora to name just a few other systems. PIE has the reputation of having a performance hit. While true, especially for register constrained architectures like i386, it makes a difference only for the executable itself, not any shared library it uses as they are already built as position independent. Also, several optimizations have been made since the early days. GCC 5 will reuse the PIC hard register (which is also good for libraries). On amd64, GCC 5 and binutils 2.26 will do copy relocations. More improvements for i386 are being worked on. I sincerly believe that users are way more likely to notice a compromised system than a slightly slower one.

Tracking progress Since version 2.5.40, lintian will now issue a warning1 when it detects that a binary has not been compiled as a position independent executable. Kudos to Niels Thykier. Now that we can track our progress, I'm calling every Debian Developers: let's try to get as many ASLR-compatible executables in Stretch!

How to enable PIE Thanks to all contributors over the past years who have improved our toolchain, we now have a fairly easy way to enable hardening flags with dpkg-buildflags. For packages using dh, it now boils down with adding on top of debian/rules:
export DEB_BUILD_MAINT_OPTIONS = hardening=+all
You can even test if a package supports all hardening flags without any changes running DEB_BUILD_OPTIONS=hardening=+all dpkg-buildpackage. Running lintian or hardening-check can then tell you if the protections have been successfully enabled.

Hardening by default? But do we really need to change so many packages individually? The difference between the current default features and all hardening features are pie and bindnow. Could we turn them by default and do binNMUs instead of requiring actions from maintainers? I guess the question boils down to: how many packages would require a (one-liner) change to turn off the pie or bindnow features if they were on by default? To get the beginning of an answer, I took the top 502 (according to popcon installations) source packages shipping non-position independent executables. I've try to rebuild them enabling all hardening flags through DEB_BUILD_OPTIONS. Out of 49 packages3:
  • 32 (65%) built fine and produced PIE binaries: acpi, bc, bind9, bsd-mailx, bsdmainutils, bzip2, cpio, cron, debianutils, diffutils, dpkg, file, fontconfig, gettext, glib2.0, glibc, gnupg, gzip, hostname, ifupdown, iputils, logrotate, m4, mutt, nano, net-tools, netcat, netkit-ftp, netkit-telnet, os-prober, pam, util-linux.
  • 4 (8%) built fine but did not compiled hardened binaries: discover, mawk, mlocate, patch.
  • 13 (27%) failed to build, with some of these expected failures, e.g. for *-static or GRUB: bash, busybox, coreutils, e2fsprogs, grub2, insserv, iptables, kbd, ncurses, newt, openssl, pciutils, perl.
The results are really encouraging. Especially taking in account that some of these packages are part of the tricky and weird set. To know for sure, we would need a mass-rebuild of the archive with DEB_BUILD_OPTIONS=hardening=+all in the environment. Any takers?

  1. Verification of the whole archive by the latest version of lintian is still in progress by the time I'm writing these lines. According to Niels it should take 3-4 more days to look at all available packages.
  2. As always, UDD does wonders:
    SELECT packages.source, MAX(popcon.insts) AS insts
      FROM lintian, popcon, packages
     WHERE lintian.tag = 'hardening-no-pie'
       AND lintian.package_arch = 'amd64'
       AND popcon.package = lintian.package
       AND packages.package = popcon.package
       AND packages.distribution = 'debian'
       AND packages.release = 'sid'
     GROUP BY packages.source
     ORDER BY MAX(popcon.insts) DESC
     LIMIT 50;
     
  3. acpid currently fail to build from source in sid.

Lunar: ALSR now!

Address space layout randomization helps to protect against buffer overflow attacks as it becomes harder for an attacker to turn a programming error into an exploitable security hole. The first implementation for Linux is 15 years old. Support in mainline kernel and toolchains have been available for a good while now. But to work, ASLR also needs executables to be built as position independent. And as Hanno B ck from the fuzzing project gently reminded me at 32C3, almost no executables in Debian are built as such, while it is now the default in Windows, Mac OS X, OpenBSD, and Fedora to name just a few other systems. PIE has the reputation of having a performance hit. While true, especially for register constrained architectures like i386, it makes a difference only for the executable itself, not any shared library it uses as they are already built as position independent. Also, several optimizations have been made since the early days. GCC 5 will reuse the PIC hard register (which is also good for libraries). On amd64, GCC 5 and binutils 2.26 will do copy relocations. More improvements for i386 are being worked on. I sincerly believe that users are way more likely to notice a compromised system than a slightly slower one.

Tracking progress Since version 2.5.40, lintian will now issue a warning1 when it detects that a binary has not been compiled as a position independent executable. Kudos to Niels Thykier. Now that we can track our progress, I'm calling every Debian Developers: let's try to get as many ALSR-compatible executables in Stretch!

How to enable PIE Thanks to all contributors over the past years who have improved our toolchain, we now have a fairly easy way to enable hardening flags with dpkg-buildflags. For packages using dh, it now boils down with adding on top of debian/rules:
export DEB_BUILD_MAINT_OPTIONS = hardening=+all
You can even test if a package supports all hardening flags without any changes running DEB_BUILD_OPTIONS=hardening=+all dpkg-buildpackage. Running lintian or hardening-check can then tell you if the protections have been successfully enabled.

Hardening by default? But do we really need to change so many packages individually? The difference between the current default features and all hardening features are pie and bindnow. Could we turn them by default and do binNMUs instead of requiring actions from maintainers? I guess the question boils down to: how many packages would require a (one-liner) change to turn off the pie or bindnow features if they were on by default? To get the beginning of an answer, I took the top 502 (according to popcon installations) source packages shipping non-position independent executables. I've try to rebuild them enabling all hardening flags through DEB_BUILD_OPTIONS. Out of 49 packages3:
  • 32 (65%) built fine and produced PIE binaries: acpi, bc, bind9, bsd-mailx, bsdmainutils, bzip2, cpio, cron, debianutils, diffutils, dpkg, file, fontconfig, gettext, glib2.0, glibc, gnupg, gzip, hostname, ifupdown, iputils, logrotate, m4, mutt, nano, net-tools, netcat, netkit-ftp, netkit-telnet, os-prober, pam, util-linux.
  • 4 (8%) built fine but did not compiled hardened binaries: discover, mawk, mlocate, patch.
  • 13 (27%) failed to build, with some of these expected failures, e.g. for *-static or GRUB: bash, busybox, coreutils, e2fsprogs, grub2, insserv, iptables, kbd, ncurses, newt, openssl, pciutils, perl.
The results are really encouraging. Especially taking in account that some of these packages are part of the tricky and weird set. To know for sure, we would need a mass-rebuild of the archive with DEB_BUILD_OPTIONS=hardening=+all in the environment. Any takers?

  1. Verification of the whole archive by the latest version of lintian is still in progress by the time I'm writing these lines. According to Niels it should take 3-4 more days to look at all available packages.
  2. As always, UDD does wonders:
    SELECT packages.source, MAX(popcon.insts) AS insts
      FROM lintian, popcon, packages
     WHERE lintian.tag = 'hardening-no-pie'
       AND lintian.package_arch = 'amd64'
       AND popcon.package = lintian.package
       AND packages.package = popcon.package
       AND packages.distribution = 'debian'
       AND packages.release = 'sid'
     GROUP BY packages.source
     ORDER BY MAX(popcon.insts) DESC
     LIMIT 50;
     
  3. acpid currently fail to build from source in sid.

25 January 2016

Steinar H. Gunderson: Chess endgame tablebases

A very short post: This link contains an interesting exposition of the 50-move rule in chess, and what it means for various endings. You can probably stop halfway, though; most of it is only interest for people deeply into endgame theory. Personally, I think DTZ50, as used by the Syzygy tablebases, is the best tradeoff for computer chess. It always produces the correct result (never throws away a win as a draw, or a draw as a loss), but the actual mates are of suboptimal length and look very strange (e.g., it will happily give away most of its pieces and then play very tricky endgames to mate). Then again, if you ever want to swindle against a non-optimal opponent (ie., try to make the position as hard as possible to play, to possibly convert e.g. a loss to a draw), you've opened up an entirely new can of worms. :-) Update: Eric P Smith has written a different explanation of the DTM50 metric that's probably easier to understand, although it contains less new information than the other one.

23 January 2016

Dirk Eddelbuettel: RcppCCTZ 0.0.3

Bradley White from the upstream CCTZ team prepared some more changes in CCTZ itself -- so a new RcppCCTZ version got to CRAN the other day catching up with these changes. CCTZ is a C++ library for translating between absolute and civil times using the rules of a time zone. It requires only a proper C++11 compiler and the standard IANA time zone data base which standard Unix, Linux, OS X, ... computers tend to have in /usr/share/zoneinfo. RcppCCTZ connects this library to R by relying on Rcpp. Changes in this version are summarized here:
Changes in version 0.0.3 (2016-01-17)
  • Synchronized with CCTZ upstream.
We also have a diff to the previous version thanks to CRANberries. More details, issue tickets etc at the GitHub repository.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

15 January 2016

Ian Wienand: Australia, ipv6 and dd-wrt

It seems that other than Internode, no Australian ISP has any details at all about native IPv6 deployment. Locally I am on Optus HFC, which I believe has been sold to the NBN, who I believe have since discovered that it is not quite what they thought it was. i.e. I think they have more problems than rolling out IPv6 and I won't hold my breath. So the only other option is to use a tunnel of some sort, and it seems there is really only one option with local presence via SixXS. There are other options, notably He.net, but they do not have Australian tunnel-servers. SixXS is the only one I could find with a tunnel in Sydney. So first sign up for an account there. The process was rather painless and my tunnel was provided quickly. After getting this, I got dd-wrt configured and working on my Netgear WNDR3700 V4. Here's my terse guide, cobbled together from other bits and pieces I found. I'm presuming you have a recent dd-wrt build that includes the aiccu tool to create the tunnel, and are pretty familiar with logging into it, etc. Firstly, on dd-wrt make sure you have JFFS2 turned on for somewhere to install scripts. Go Administration, JFFS2 Support, Internal Flash Storage, Enabled. Next, add the aiccu config file to /jffs/etc/aiccu.conf
# AICCU Configuration
# Login information
username USERNAME
password PASSWORD
# Protocol and server listed on your tunnel
protocol tic
server tic.sixxs.net
# Interface names to use
ipv6_interface sixxs
# The tunnel_id to use
# (only required when there are multiple tunnels in the list)
#tunnel_id <your tunnel id>
# Be verbose?
verbose false
# Daemonize?
daemonize true
# Require TLS?
requiretls true
# Set default route?
defaultroute true
Now you can add a script to bring up the tunnel and interface to /jffs/config/sixxs.ipup (make sure you make it executable) where you replace your tunnel address in the ip commands.
# wait until time is synced
while [  date +%Y  -eq 1970 ]; do
sleep 5
done
# check if aiccu is already running
if [ -n " ps grep etc/aiccu grep -v grep " ]; then
aiccu stop
sleep 1
killall aiccu
fi
# start aiccu
sleep 3
aiccu start /jffs/etc/aiccu.conf
sleep 3
ip -6 addr add 2001:....:....:....::/64 dev br0
ip -6 route add 2001:....:....:....::/64 dev br0
sleep 5
#### BEGIN FIREWALL RULES ####
WAN_IF=sixxs
LAN_IF=br0
#flush tables
ip6tables -F
#define policy
ip6tables -P INPUT DROP
ip6tables -P FORWARD DROP
ip6tables -P OUTPUT ACCEPT
# Input to the router
# Allow all loopback traffic
ip6tables -A INPUT -i lo -j ACCEPT
#Allow unrestricted access on internal network
ip6tables -A INPUT -i $LAN_IF -j ACCEPT
#Allow traffic related to outgoing connections
ip6tables -A INPUT -i $WAN_IF -m state --state RELATED,ESTABLISHED -j ACCEPT
# for multicast ping replies from link-local addresses (these don't have an
# associated connection and would otherwise be marked INVALID)
ip6tables -A INPUT -p icmpv6 --icmpv6-type echo-reply -s fe80::/10 -j ACCEPT
# Allow some useful ICMPv6 messages
ip6tables -A INPUT -p icmpv6 --icmpv6-type destination-unreachable -j ACCEPT
ip6tables -A INPUT -p icmpv6 --icmpv6-type packet-too-big -j ACCEPT
ip6tables -A INPUT -p icmpv6 --icmpv6-type time-exceeded -j ACCEPT
ip6tables -A INPUT -p icmpv6 --icmpv6-type parameter-problem -j ACCEPT
ip6tables -A INPUT -p icmpv6 --icmpv6-type echo-request -j ACCEPT
ip6tables -A INPUT -p icmpv6 --icmpv6-type echo-reply -j ACCEPT
# Forwarding through from the internal network
# Allow unrestricted access out from the internal network
ip6tables -A FORWARD -i $LAN_IF -j ACCEPT
# Allow some useful ICMPv6 messages
ip6tables -A FORWARD -p icmpv6 --icmpv6-type destination-unreachable -j ACCEPT
ip6tables -A FORWARD -p icmpv6 --icmpv6-type packet-too-big -j ACCEPT
ip6tables -A FORWARD -p icmpv6 --icmpv6-type time-exceeded -j ACCEPT
ip6tables -A FORWARD -p icmpv6 --icmpv6-type parameter-problem -j ACCEPT
ip6tables -A FORWARD -p icmpv6 --icmpv6-type echo-request -j ACCEPT
ip6tables -A FORWARD -p icmpv6 --icmpv6-type echo-reply -j ACCEPT
#Allow traffic related to outgoing connections
ip6tables -A FORWARD -i $WAN_IF -m state --state RELATED,ESTABLISHED -j ACCEPT
Now you can reboot, or run the script, and it should bring the tunnel up and you should be correclty firewalled such that packets get out, but nobody can get in. Back to the web-interface, you can now enable IPv6 with Setup, IPV6, Enable. You leave "IPv6 Type" as Native IPv6 from ISP. Then I enabled Radvd and added a custom config in the text-box to get DNS working with google DNS on hosts with:
interface br0
 
AdvSendAdvert on;
prefix 2001:....:....:....::/64
  
  ;
 RDNSS 2001:4860:4860::8888 2001:4860:4860::8844
  
  ;
 ;
(again, replace the prefix with your own) That is pretty much it; at this point, you should have an IPv6 network and it's most likely that all your network devices will "just work" with it. I got full scores on the IPv6 test sites on a range of devices. Unfortunately, even a geographically close tunnel still really kills latency; compare these two traceroutes:
$ mtr -r -c 1 google.com
Start: Fri Jan 15 14:51:18 2016
HOST: jj                          Loss%   Snt   Last   Avg  Best  Wrst StDev
1.  -- 2001:....:....:....::      0.0%     1    1.4   1.4   1.4   1.4   0.0
2.  -- gw-163.syd-01.au.sixxs.ne  0.0%     1   12.0  12.0  12.0  12.0   0.0
3.  -- ausyd01.sixxs.net          0.0%     1   13.5  13.5  13.5  13.5   0.0
4.  -- sixxs.sydn01.occaid.net    0.0%     1   13.7  13.7  13.7  13.7   0.0
5.  -- 15169.syd.equinix.com      0.0%     1   11.5  11.5  11.5  11.5   0.0
6.  -- 2001:4860::1:0:8613        0.0%     1   14.1  14.1  14.1  14.1   0.0
7.  -- 2001:4860::8:0:79a0        0.0%     1  115.1 115.1 115.1 115.1   0.0
8.  -- 2001:4860::8:0:8877        0.0%     1  183.6 183.6 183.6 183.6   0.0
9.  -- 2001:4860::1:0:66d6        0.0%     1  196.6 196.6 196.6 196.6   0.0
10. -- 2001:4860:0:1::72d         0.0%     1  189.7 189.7 189.7 189.7   0.0
11. -- kul01s07-in-x09.1e100.net  0.0%     1  194.9 194.9 194.9 194.9   0.0
$ mtr -4 -r -c 1 google.com
Start: Fri Jan 15 14:51:46 2016
HOST: jj                          Loss%   Snt   Last   Avg  Best  Wrst StDev
1. -- gateway                    0.0%     1    1.3   1.3   1.3   1.3   0.0
2. -- 10.50.0.1                  0.0%     1   11.0  11.0  11.0  11.0   0.0
3. -- ???                       100.0     1    0.0   0.0   0.0   0.0   0.0
4. -- ???                       100.0     1    0.0   0.0   0.0   0.0   0.0
5. -- ???                       100.0     1    0.0   0.0   0.0   0.0   0.0
6. -- riv4-ge4-1.gw.optusnet.co  0.0%     1   12.1  12.1  12.1  12.1   0.0
7. -- 198.142.187.20             0.0%     1   10.4  10.4  10.4  10.4   0.0
When you watch what is actually using ipv6 (the ipvfoo plugin for Chrome is pretty cool, it shows you what requests are going where), it's mostly all just traffic to really big sites (Google/Google Analytics, Facebook, Youtube, etc) who have figured out IPv6. Since these are exactly the type of places that have made efforts to get caching as close as possible to you (Google's mirror servers are within Optus' network, for example) and so you're really shooting yourself in the foot going around it using an external tunnel. The other thing is that I'm often hitting IPv6 mirrors and downloading larger things for work stuff (distro updates, git clones, image downloads, etc) which is slower and wasting someone else's bandwith for really no benefit. So while it's pretty cool to have an IPv6 address (and a fun experiment) I think I'm going to turn it off. One positive was that after running with it for about a month, nothing has broken -- which suggests that most consumer level gear in a typical house (phones, laptops, TVs, smart-watches, etc) is either ready or ignores it gracefully. Bring on native IPv6!

14 January 2016

Rapha&#235;l Hertzog: Working as a paid LTS contributor

A Debian LTS logoWhile the details about how to join the set of paid contributors have always been public (here) we did not advertise this fact very much outside of the people already interested in LTS (and thus subscribed to debian-lts@lists.debian.org). But right now we would like to have a few more paid contributors on board and I m thus posting this call for volunteers. Who can apply? You need to meet those requirements:
  • you are Debian Developer or a Debian Maintainer;
  • you have some prior experience with providing security updates in Debian (at least on your own packages);
  • you have good programming skills and know multiple languages (to be able to backport security fixes);
  • you can emit invoices to Freexian;
  • you accept the rules defined for this project:
    • you must respect the privacy of any customer data;
    • you must prepare a public monthly report of the work done on paid time;
    • you must respect the Debian code of conduct and respond to queries about your work from fellow community members;
    • you must do your best to meet the high-quality standards set by the Debian security team.
Even though Freexian is located in France and requires you to provide invoice in EUR, there are no conditions on your nationality or country of residence. For contributors outside of the Euro zone, Freexian is using Transferwise to pay them with minimal currency conversion costs (Paypal is also possible if nothing else works). The rate offered to paid contributors is the same for all (75 EUR/hour), it s based on a correct rate for independent contractors in western Europe. If the rate is very high for your own country, then be happy to be able to invoice Freexian at this rate and use this opportunity to work less (for money) and contribute more to Debian on your (now copious) free time. How does the work look like? If you apply, you will have to send us an SSH key so that you can have access to the internal git repository used for work. It contains a ledger file to track the hours funded by sponsors and how they have been dispatched to the various contributors. You can always know how many hours are assigned to you, how many can be invoiced, and so on. You will have to update it once a month to record the work you did (and indicate us where the report has been published). The repository also contains a README with many explanations about the workflow (how hours are dispatched, the delay you have to publish your report, etc) and a small helper script (./find-work) to match up the pending updates (registered in dla-needed.txt) with the popularity of the package among the sponsors. Now the work itself is relatively well documented in the LTS wiki. You will have to provide updates for packages that need an update. You have some freedom in selecting the packages but at some point you will have to work on packages that you don t know that are written in a language that you have almost not used. So you must be able to go out of your comfort zone and still do a good work. You must also be able to multi-task because in some cases you will get stuck on a particular update and you will have to seek help from the upstream developer (or from the Debian package maintainer). Don t expect to be able to do all your work hours in a single run thus don t wait until the last days of the month. Start early and dispatch your work hours over the month. From time to time, you will also have to handle the LTS frontdesk for one week. During this week, you need to spend a bit of time every day to triage the new CVE, to respond to questions on the mailing list, and to sponsor updates prepared by volunteers who do not have upload rights. Questions? Ask your questions in the comments and I will update this section with your questions and our answers. What if I have no prior experience with security updates? Start getting some experience. The LTS and security teams are open for anyone to join. Read their documentation and provide some updates that other contributors can sponsor. Before accepting you as paid contributor, we generally ask you to prepare one or two DLA on your free time just to make sure that you know the workflow and that you are up to the task. What if I have only X hours available for paid LTS work? In the git repository there s a file where you document how many work hours you can handle. You might get less than this amount, but we generally never assign less than 8 hours (to make sure that you can handle one complicated update from start to end, or your possible week of LTS frontdesk). You can adjust it each month or even opt-out if you are not available for whatever reason. But once you have been assigned work hours, it s important to actually do the work that you requested! How do I apply? Get in touch with me (as documented).

One comment Liked this article? Click here. My blog is Flattr-enabled.

13 January 2016

Norbert Preining: Ian Buruma: Wages of Guilt

Since moving to Japan, I got more and more interested in history, especially the recent history of the 20th century. The book I just finished, Ian Buruma (Wiki, home page) Wages of Guilt Memories of War in Germany and Japan (Independent, NYRB), has been a revelation for me. As an Austrian living in Japan, I am experiencing the discrepancy between these two countries with respect to their treatment of war legacy practically daily, and many of my blog entries revolve around the topic of Japanese non-reconciliation.
Willy Brandt went down on his knees in the Warsaw ghetto, after a functioning democracy had been established in the Federal Republic of Germany, not before. But Japan, shielded from the evil world, has grown into an Oskar Matzerath: opportunistic, stunted, and haunted by demons, which it tries to ignore by burying them in the sand, like Oskar s drum.
Ian Buruma, Wages of Guilt, Clearing Up the Ruins
Buruma-Wages_of_Guilt The comparison of Germany and Japan with respect to their recent history as laid out in Buruma s book throws a spotlight on various aspects of the psychology of German and Japanese population, while at the same time not falling into the easy trap of explaining everything with difference in the guilt culture. A book of great depth and broad insights everyone having even the slightest interest in these topics should read.
This difference between (West) German and Japanese textbooks is not just a matter of detail; it shows a gap in perception.
Ian Buruma, Wages of Guilt, Romance of the Ruins
Only thinking about giving a halfway full account of this book is something impossible for me. The sheer amount of information, both on the German and Japanese side, is impressive. His incredible background (studies of Chinese literature and Japanese movie!) and long years as journalist, editor, etc, enriches the book with facets normally not available: In particular his knowledge of both the German and Japanese movie history, and the reflection of history in movies, were complete new aspects for me (see my recent post (in Japanese)). The book is comprised of four parts: The first with the chapters War Against the West and Romance of the Ruins; the second with the chapters Auschwitz, Hiroshima, and Nanking; the third with History on Trial, Textbook Resistance, and Memorials, Museums, and Monuments; and the last part with A Normal Country, Two Normal Towns, and Clearing Up the Ruins. Let us look at the chapters in turn:
  • War Against the West
    This chapter sets the stage in two parts, Bonn and Tokyo, by comparing the reaction in these countries to the Iraq war. The German Betroffenheit (To be betroffen implies a sense of guilt, a sense of shame, or even embarrassment.) as the core of German post-war politics, literature, and media is introduced. On the Japanese side the difficult and diverse situation and attitudes towards the Iraq (and other) wars, as well as the necessary bits of post-war history and development of the Japanese constitution.
    What is so convenient in the cases of Germany and Japan is that pacifism happens to be a high-minded way to dull the pain of historical guilt. Or, conversely, if one wallows in it, pacifism turns national guilt into a virtue, almost a mark of superiority, when compared to the complacency of other nations.
  • Romance of the Ruins
    This chapter focuses on the war and immediate post-war period with references to the specific literature and movies emerging out of the circumstances of destroyed countries who have lost the war.
    Hitler s doom and the emperor s speech, the end of one symbol and the odd continuity of another. Whatever their symbolic differences, both would be associated forever with ruins ruined cities, ruined people, ruined ideals.
  • Auschwitz
    The psychological construction of war memorials in both Germanies, which focuses on the religious aspects, is discussed, followed by an excursion through post-war German literature and the long-term ignorance of anything related to the Holocaust.
    Here the past had fossilized into something monumental or, as Adorno would have put it, museal.
  • Hiroshima Paralleling the previous chapter, Hiroshima introduces the simplistic and reduced focus of the Hiroshima memorials, mostly ignoring the foreign victims, many of them being Koreans forced to work in Japan, and concentrating on the Japanese martyrdom. Focusing on the atomic bomb event everything else is removed from the field of view.
    The problem with this quasi-religious view of history is that it makes it hard to discuss past events in anything but nonsecular terms. Visions of absolute evil are unique, and they are beyond human explanation or even comprehension. To explain is hubristic and amoral. If this is true of Auschwitz, it is even more true of Hiroshima. The irony is that while there can be no justification for Auschwitz unless one believes in Hitler s murderous ideology, the case for Hiroshima is at least open to debate. The A-bomb might have saved lives; it might have shortened the war. But such arguments are incompatible with the Hiroshima spirit.
  • Nanking
    The history and aftermath, as well as the attempts of rejection and refutation of the Nanking massacre are described. The Tokyo Trials and their critique by governmental scholars are touched, as well as bit of fresh air blowing through the Japanese society after the death of Hirohito, which lead to the publication of the records of Nanking by Azuma Shiro .
    Yet the question remains whether the raping and killing of thousands of women, and the massacre of thousands, perhaps hundreds of thousands, of other unarmed people, in the course of six weeks, can still be called extreme conduct in the heat of battle. The question is pertinent, particularly when such extreme violence is justified by an ideology which teaches the aggressors that killing an inferior race is in accordance with the will of their divine emperor.
  • History on Trial
    One of the central chapters in my opinion. It discusses and compares the two post-war trials: The Nurnberg trials in Germany and the Tokyo trials in Japan. In both cases the juridical value is questioned, focusing on the winner-looser situation of post-war times.
    The Nuremberg trials were to be a history lesson, then, as well as a symbolic punishment of the German people a moral history lesson cloaked in all the ceremonial trappings of due legal process. They were the closest that man, or at least the men belonging to the victorious powers, could come to dispensing divine justice.
    Also, the differences in war trials in East and West Germany is compared. The East Germany Waldheimer trials, as well as the thorough purge of Nazis from East German jurisdiction and politics, which was in stark contrast to both West German s very restricted trials, as well as Japan s absolute non-purge of criminals.
    As long as the emperor lived, Japanese would have trouble being honest about the past. For he had been formally responsible for everything, and by holding him responsible for nothing, everybody was absolved, except, of course, for a number of military and civilian scapegoats, Officers and Outlaws, who fell victim to victors justice.
  • Textbook Resistance This chapter compares the representation of war and post-war times in the textbooks in West and East Germany and Japan. The interesting case of Ienaga Saburo and the year-long trials (1965-1993) around his history textbook are recounted. The ministry of education had forced a redaction of his history textbook to conform with the revisionist view onto history, deleting most passages that are critical of the Japanese position during the first half of the 20th century. This was one of the very few cases in Japanese post-war history where someone stood up against this revisionist view.
    The judges and some of the counsel for the ministry sat back with their eyes closed, in deep concentration, or fast asleep. Perhaps they were bored, because they had heard it all before. Perhaps they thought it was a pointless exercise, since they knew already how the case would end. But it was not a pointless exercise. For Ienaga Saburo had kept alive a vital debate for twenty-seven years. One cussed schoolteacher and several hundred supporters at the courthouse might not seem much, but it was enough to show that, this time, someone was fighting back.
  • Memorials, Museums, and Monuments
    This chapter returns to war memorials: The change of meaning from post WW-1, which were memorials, to post WW-2 ones which became warning monuments, indicating the shift of attention and evaluation of war history in Germany. In contrast to this, Japan s quasi non-existence of war museums till the late 90ies, as well as the existence of the Yasukuni shrine honoring and celebrating besides other several A-class war criminals as deities.
    The tragedy is not just that the suicide pilots died young. Soldiers (and civilians) do that in wars everywhere. What is so awful about the memory of their deaths is the cloying sentimentality that was meant to justify their self-immolation. There is no reason to suppose they didn t believe in the patriotic gush about cherry blossoms and sacrifice, no matter how conventional it was at the time. Which was exactly the point: they were made to rejoice in their own death. It was the exploitation of their youthful idealism that made it such a wicked enterprise. And this point is still completely missed at the Peace Museum today.
  • A Normal Country This chapter discusses the slow normalization of post-war situation after the 90ies, and all the hurdles that needed to be overcome: In the case of Germany the speech of Philipp Jenninger, then president of the Bundestag, is recounted. 50 years after the Kristallnacht he tried to give a speech of historicization , only to be find himself shunned and expelled due to the lack of Betroffenheit.
    It was not an ignoble enterprise, but he should have recognized that Historisierung, even forty-three years after the war, was still a highly risky business. For a normal society, a society not haunted by ghosts, cannot be achieved by normalizing history, or by waving cross and garlic. More the other way around: when society has become sufficiently open and free to look back, from the point of view neither of the victim nor of the criminal, but of the critic, only then will the ghosts be laid to rest.
    On the Japanese side the case of Motoshima Hitoshi , who dared to question Hirohito:
    Forty-three years have passed since the end of the war, and I think we have had enough chance to reflect on the nature of that war. From reading various accounts from abroad and having been a soldier myself, involved in military education, I do believe that the emperor bore responsibility for the war.
    which led to hitherto unseen of demonstration of extreme-right-wing groups issuing death treats that lead to a failed assassination of Motoshima, all under the completely complacent Japanese police and politics letting the right-wingers play their game.
    By breaking a Japanese taboo, Motoshima struck a blow for a more open, more normal political society, and very nearly lost his life. Jenninger, I like to think, wanted to strike a blow for the same, but failed, and lost his job. Perhaps he wasn t up to the task. Or perhaps even West Germany was not yet normal enough to hear his message.
  • Two Normal Towns This chapter focuses on two rare cases of civil courage and political commitment: Anja Rosmus, who stepped forth as school child to rewrite the history of Passau. She unveiled the truth about deep involvement into the NS crimes of many inhabitants of Passau, a fact that was up to then covered up and purged from knowledge. She, too, received many death threads, including nailing a killed cat onto her door. The response of the head of the tourist office in Passau, Gottfried Dominik, speaks about the very peculiar attitude:
    I asked him again about the local camp and the small hidden memorial. Dominik showed signs of distress. It was difficult, he admitted, very difficult. I know what you mean. But let me give you my personal opinion. When you have a crippled arm, you don t really want to show it around. It was a low point in our history, back then. But it was only twelve years in thousands of years of history. And so people tend to hide it, just as a person with a crippled arm is not likely to wear a short-sleeved shirt.
    A similar incident is recounted on the Japanese side, the Hanaoka incident (detailed article) and its unveiling by Nozoe Kenji, where 800 Chinese slave workers, after escaping from a forced-work camp for the Kajima Corporation, where rabbit-hunted down and slaughtered. He, too, got death threats, and was virtually expelled from his home area because he dared to publish his findings.
    I think it is this basic distrust, this refusal to be told what to think by authorities, this cussed insistence on asking questions, on hearing the truth, that binds together Nozoe, Rosmus, and others like them. There are not many such people in Japan, or anywhere else for that matter. And I suspect they are not much liked wherever they live.
  • Clearing Up the Ruins The last chapter tries to round up all the previous chapters, and look into the most recent history and near future. While not completely pessimistic with respect to Japan, the final chapter leaves clear statements on the current state of Japanese society and politics:
    The state was run by virtually the same bureaucracy that ran the Japanese empire, and the electoral system was rigged to help the same corrupt conservative party to stay in power for almost forty years. This arrangement suited the United States, as well as Japanese bureaucrats, LDP politicians, and the large industrial combines, for it ensured that Japan remained a rich and stable ally against Communism. But it also helped to stifle public debate and stopped the Japanese from growing up politically.
    His description of current Japanese society, written in 1995, is still hauntingly true in 2016:
    There is something intensely irritating about the infantilism of postwar Japanese culture: the ubiquitous chirping voices of women pretending to be girls; the Disneylandish architecture of Japanese main streets, where everything is reduced to a sugary cuteness; the screeching television talents rolling about and carrying on like kindergarten clowns; the armies of blue-suited salarymen straphanging on the subway trains, reading boys comics, the maudlin love for old school songs and cuddly mama-sans.
The boook somehow left me with a bleak impression of Japanese post-war times as well as Japanese future. Having read other books about the political ignorance in Japan (Norma Field s In the realm of a dying emperor, or the Chibana history), Buruma s characterization of Japanese politics is striking. He couldn t foresee the recent changes in legislation pushed through by the Abe government actually breaking the constitution, or the rewriting of history currently going on with respect to comfort women and Nanking. But reading his statement about Article Nine of the constitution and looking at the changes in political attitude, I am scared about where Japan is heading to:
The Nanking Massacre, for leftists and many liberals too, is the main symbol of Japanese militarism, supported by the imperial (and imperialist) cult. Which is why it is a keystone of postwar pacifism. Article Nine of the constitution is necessary to avoid another Nanking Massacre. The nationalist right takes the opposite view. To restore the true identity of Japan, the emperor must be reinstated as a religious head of state, and Article Nine must be revised to make Japan a legitimate military power again. For this reason, the Nanking Massacre, or any other example of extreme Japanese aggression, has to be ignored, softened, or denied.
Ian Buruma, Wages of Guilt, Nanking
While there are signs of resistance in the streets of Japan (Okinawa and the Hanako bay, the demonstrations against secrecy law and reversion of the constitution), we are still to see a change influenced by the people in a country ruled and distributed by oligarchs. I don t think there will be another Nanking Massacre in the near future, but Buruma s books shows that we are heading back to a nationalistic regime similar to pre-war times, just covered with a democratic veil to distract critics.
I close with several other quotes from the book that caught my attention: In the preface and introduction:
[ ] mainstream conservatives made a deliberate attempt to distract people s attention from war and politics by concentrating on economic growth.
The curious thing was that much of what attracted Japanese to Germany before the war Prussian authoritarianism, romantic nationalism, pseudo-scientific racialism had lingered in Japan while becoming distinctly unfashionable in Germany.
In Romance of the Ruins:
The point of all this is that Ikeda s promise of riches was the final stage of what came to be known as the reverse course, the turn away from a leftist, pacifist, neutral Japan a Japan that would never again be involved in any wars, that would resist any form of imperialism, that had, in short, turned its back for good on its bloody past. The Double Your Incomes policy was a deliberate ploy to draw public attention away from constitutional issues.
In Hiroshima:
The citizens of Hiroshima were indeed victims, primarily of their own military rulers. But when a local group of peace activists petitioned the city of Hiroshima in 1987 to incorporate the history of Japanese aggression into the Peace Memorial Museum, the request was turned down. The petition for an Aggressors Corner was prompted by junior high school students from Osaka, who had embarrassed Peace Museum officials by asking for an explanation about Japanese responsibility for the war.
The history of the war, or indeed any history, is indeed not what the Hiroshima spirit is about. This is why Auschwitz is the only comparison that is officially condoned. Anything else is too controversial, too much part of the flow of history .
In Nanking, by the governmental pseudo-historian Tanaka:
Unlike in Europe or China, writes Tanaka, you won t find one instance of planned, systematic murder in the entire history of Japan. This is because the Japanese have a different sense of values from the Chinese or the Westerners.
In History on Trial:
In 1950, Becker wrote that few things have done more to hinder true historical self-knowledge in Germany than the war crimes trials. He stuck to this belief. Becker must be taken seriously, for he is not a right-wing apologist for the Nazi past, but an eminent liberal.
There never were any Japanese war crimes trials, nor is there a Japanese Ludwigsburg. This is partly because there was no exact equivalent of the Holocaust. Even though the behavior of Japanese troops was often barbarous, and the psychological consequences of State Shinto and emperor worship were frequently as hysterical as Nazism, Japanese atrocities were part of a military campaign, not a planned genocide of a people that included the country s own citizens. And besides, those aspects of the war that were most revolting and furthest removed from actual combat, such as the medical experiments on human guinea pigs (known as logs ) carried out by Unit 731 in Manchuria, were passed over during the Tokyo trial. The knowledge compiled by the doctors of Unit 731 of freezing experiments, injection of deadly diseases, vivisections, among other things was considered so valuable by the Americans in 1945 that the doctors responsible were allowed to go free in exchange for their data.
Some Japanese have suggested that they should have conducted their own war crimes trials. The historian Hata Ikuhiko thought the Japanese leaders should have been tried according to existing Japanese laws, either in military or in civil courts. The Japanese judges, he believed, might well have been more severe than the Allied tribunal in Tokyo. And the consequences would have been healthier. If found guilty, the spirits of the defendants would not have ended up being enshrined at Yasukuni. The Tokyo trial, he said, purified the crimes of the accused and turned them into martyrs. If they had been tried in domestic courts, there is a good chance the real criminals would have been flushed out.
After it was over, the Nippon Times pointed out the flaws of the trial, but added that the Japanese people must ponder over why it is that there has been such a discrepancy between what they thought and what the rest of the world accepted almost as common knowledge. This is at the root of the tragedy which Japan brought upon herself.
Emperor Hirohito was not Hitler; Hitler was no mere Shrine. But the lethal consequences of the emperor-worshipping system of irresponsibilities did emerge during the Tokyo trial. The savagery of Japanese troops was legitimized, if not driven, by an ideology that did not include a Final Solution but was as racialist as Hider s National Socialism. The Japanese were the Asian Herrenvolk, descended from the gods.
Emperor Hirohito, the shadowy figure who changed after the war from navy uniforms to gray suits, was not personally comparable to Hitler, but his psychological role was remarkably similar.
In fact, MacArthur behaved like a traditional Japanese strongman (and was admired for doing so by many Japanese), using the imperial symbol to enhance his own power. As a result, he hurt the chances of a working Japanese democracy and seriously distorted history. For to keep the emperor in place (he could at least have been made to resign), Hirohito s past had to be freed from any blemish; the symbol had to be, so to speak, cleansed from what had been done in its name.
In Memorials, Museums, and Monuments:
If one disregards, for a moment, the differences in style between Shinto and Christianity, the Yasukuni Shrine, with its relics, its sacred ground, its bronze paeans to noble sacrifice, is not so very different from many European memorials after World War I. By and large, World War II memorials in Europe and the United States (though not the Soviet Union) no longer glorify the sacrifice of the fallen soldier. The sacrificial cult and the romantic elevation of war to a higher spiritual plane no longer seemed appropriate after Auschwitz. The Christian knight, bearing the cross of king and country, was not resurrected. But in Japan, where the war was still truly a war (not a Holocaust), and the symbolism still redolent of religious exultation, such shrines as Yasukuni still carry the torch of nineteenth-century nationalism. Hence the image of the nation owing its restoration to the sacrifice of fallen soldiers.
In A Normal Country:
The mayor received a letter from a Shinto priest in which the priest pointed out that it was un-Japanese to demand any more moral responsibility from the emperor than he had already taken. Had the emperor not demonstrated his deep sorrow every year, on the anniversary of Japan s surrender? Besides, he wrote, it was wrong to have spoken about the emperor in such a manner, even as the entire nation was deeply worried about his health. Then he came to the main point: It is a common error among Christians and people with Western inclinations, including so-called intellectuals, to fail to grasp that Western societies and Japanese society are based on fundamentally different religious concepts . . . Forgetting this premise, they attempt to place a Western structure on a Japanese foundation. I think this kind of mistake explains the demand for the emperor to bear full responsibility.
In Two Normal Towns:
The bust of the man caught my attention, but not because it was in any way unusual; such busts of prominent local figures can be seen everywhere in Japan. This one, however, was particularly grandiose. Smiling across the yard, with a look of deep satisfaction over his many achievements, was Hatazawa Kyoichi. His various functions and titles were inscribed below his bust. He had been an important provincial bureaucrat, a pillar of the sumo wrestling establishment, a member of various Olympic committees, and the recipient of some of the highest honors in Japan. The song engraved on the smooth stone was composed in praise of his rich life. There was just one small gap in Hatazawa s life story as related on his monument: the years from 1941 to 1945 were missing. Yet he had not been idle then, for he was the man in charge of labor at the Hanaoka mines.
In Clearing Up the Ruins:
But the question in American minds was understandable: could one trust a nation whose official spokesmen still refused to admit that their country had been responsible for starting a war? In these Japanese evasions there was something of the petulant child, stamping its foot, shouting that it had done nothing wrong, because everybody did it.
Japan seems at times not so much a nation of twelve-year-olds, to repeat General MacArthur s phrase, as a nation of people longing to be twelve-year-olds, or even younger, to be at that golden age when everything was secure and responsibility and conformity were not yet required.
For General MacArthur was right: in 1945, the Japanese people were political children. Until then, they had been forced into a position of complete submission to a state run by authoritarian bureaucrats and military men, and to a religious cult whose high priest was also formally chief of the armed forces and supreme monarch of the empire.
I saw Jew S ss that same year, at a screening for students of the film academy in Berlin. This showing, too, was followed by a discussion. The students, mostly from western Germany, but some from the east, were in their early twenties. They were dressed in the international uniform of jeans, anoraks, and work shirts. The professor was a man in his forties, a 68er named Karsten Witte. He began the discussion by saying that he wanted the students to concentrate on the aesthetics of the film more than the story. To describe the propaganda, he said, would simply be banal: We all know the what, so let s talk about the how. I thought of my fellow students at the film school in Tokyo more than fifteen years before. How many of them knew the what of the Japanese war in Asia.

Next.

Previous.