Because posting private keys on the Internet is a bad idea, some people like to redact their private keys, so that it looks kinda-sorta like a private key, but it isn t actually giving away anything secret. Unfortunately, due to the way that private keys are represented, it is easy to redact a key in such a way that it doesn t actually redact anything at all. RSA private keys are particularly bad at this, but the problem can (potentially) apply to other keys as well. I ll show you a bit of Inside Baseball with key formats, and then demonstrate the practical implications. Finally, we ll go through a practical worked example from an actual not-really-redacted key I recently stumbled across in my travels.

The Private Lives of Private Keys Here is what a typical private key looks like, when you come across it:

-----BEGIN RSA PRIVATE KEY-----
MGICAQACEQCxjdTmecltJEz2PLMpS4BXAgMBAAECEDKtuwD17gpagnASq1zQTYEC
CQDVTYVsjjF7IQIJANUYZsIjRsR3AgkAkahDUXL0RSECCB78r2SnsJC9AghaOK3F
sKoELg==
-----END RSA PRIVATE KEY-----

Obviously, there s some hidden meaning in there computers don t encrypt things by shouting BEGIN RSA PRIVATE KEY! , after all. What is between the BEGIN/END lines above is, in fact, a base64-encoded DER format ASN.1 structure representing a PKCS#1 private key. In simple terms, it s a list of numbers very important numbers. The list of numbers is, in order:

• A version number (0);
• The public modulus , commonly referred to as n ;
• The public exponent , or e (which is almost always 65,537, for various unimportant reasons);
• The private exponent , or d ;
• The two private primes , or p and q ;
• Two exponents, which are known as dmp1 and dmq1 ; and
• A coefficient, known as iqmp .

Why Is This a Problem? The thing is, only three of those numbers are actually required in a private key. The rest, whilst useful to allow the RSA encryption and decryption to be more efficient, aren t necessary. The three absolutely required values are e, p, and q. Of the other numbers, most of them are at least about the same size as each of p and q. So of the total data in an RSA key, less than a quarter of the data is required. Let me show you with the above toy key, by breaking it down piece by piece¹:

• MGI DER for this is a sequence
• CAQ version (0)
• CxjdTmecltJEz2PLMpS4BX n
• AgMBAA e
• ECEDKtuwD17gpagnASq1zQTY d
• ECCQDVTYVsjjF7IQ p
• IJANUYZsIjRsR3 q
• AgkAkahDUXL0RS dmp1
• ECCB78r2SnsJC9 dmq1
• AghaOK3FsKoELg== iqmp

Remember that in order to reconstruct all of these values, all I need are e, p, and q and e is pretty much always 65,537. So I could redact almost all of this key, and still give all the important, private bits of this key. Let me show you:

-----BEGIN RSA PRIVATE KEY-----
..............................................................EC
CQDVTYVsjjF7IQIJANUYZsIjRsR3....................................
........
-----END RSA PRIVATE KEY-----

Now, I doubt that anyone is going to redact a key precisely like this but then again, this isn t a typical RSA key. They usually look a lot more like this:

-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAu6Inch7+mWtKn+leB9uCG3MaJIxRyvC/5KTz2fR+h+GOhqj4
SZJobiVB4FrE5FgC7AnlH6qeRi9MI0s6dt5UWZ5oNIeWSaOOeNO+EJDUkSVf67wj
SNGXlSjGAkPZ0nRJiDjhuPvQmdW53hOaBLk5udxPEQbenpXAzbLJ7wH5ouLQ3nQw
HwpwDNQhF6zRO8WoscpDVThOAM+s4PS7EiK8ZR4hu2toon8Ynadlm95V45wR0VlW
zywgbkZCKa1IMrDCscB6CglQ10M3Xzya3iTzDtQxYMVqhDrA7uBYRxA0y1sER+Rb
yhEh03xz3AWemJVLCQuU06r+FABXJuY/QuAVvQIDAQABAoIBAFqwWVhzWqNUlFEO
PoCVvCEAVRZtK+tmyZj9kU87ORz8DCNR8A+/T/JM17ZUqO2lDGSBs9jGYpGRsr8s
USm69BIM2ljpX95fyzDjRu5C0jsFUYNi/7rmctmJR4s4uENcKV5J/++k5oI0Jw4L
c1ntHNWUgjK8m0UTJIlHbQq0bbAoFEcfdZxd3W+SzRG3jND3gifqKxBG04YDwloy
tu+bPV2jEih6p8tykew5OJwtJ3XsSZnqJMwcvDciVbwYNiJ6pUvGq6Z9kumOavm9
XU26m4cWipuK0URWbHWQA7SjbktqEpxsFrn5bYhJ9qXgLUh/I1+WhB2GEf3hQF5A
pDTN4oECgYEA7Kp6lE7ugFBDC09sKAhoQWrVSiFpZG4Z1gsL9z5YmZU/vZf0Su0n
9J2/k5B1GghvSwkTqpDZLXgNz8eIX0WCsS1xpzOuORSNvS1DWuzyATIG2cExuRiB
jYWIJUeCpa5p2PdlZmBrnD/hJ4oNk4oAVpf+HisfDSN7HBpN+TJfcAUCgYEAyvY7
Y4hQfHIdcfF3A9eeCGazIYbwVyfoGu70S/BZb2NoNEPymqsz7NOfwZQkL4O7R3Wl
Rm0vrWT8T5ykEUgT+2ruZVXYSQCKUOl18acbAy0eZ81wGBljZc9VWBrP1rHviVWd
OVDRZNjz6nd6ZMrJvxRa24TvxZbJMmO1cgSW1FkCgYAoWBd1WM9HiGclcnCZknVT
UYbykCeLO0mkN1Xe2/32kH7BLzox26PIC2wxF5seyPlP7Ugw92hOW/zewsD4nLze
v0R0oFa+3EYdTa4BvgqzMXgBfvGfABJ1saG32SzoWYcpuWLLxPwTMsCLIPmXgRr1
qAtl0SwF7Vp7O/C23mNukQKBgB89DOEB7xloWv3Zo27U9f7nB7UmVsGjY8cZdkJl
6O4LB9PbjXCe3ywZWmJqEbO6e83A3sJbNdZjT65VNq9uP50X1T+FmfeKfL99X2jl
RnQTsrVZWmJrLfBSnBkmb0zlMDAcHEnhFYmHFuvEnfL7f1fIoz9cU6c+0RLPY/L7
n9dpAoGAXih17mcmtnV+Ce+lBWzGWw9P4kVDSIxzGxd8gprrGKLa3Q9VuOrLdt58
++UzNUaBN6VYAe4jgxGfZfh+IaSlMouwOjDgE/qzgY8QsjBubzmABR/KWCYiRqkj
qpWCgo1FC1Gn94gh/+dW2Q8+NjYtXWNqQcjRP4AKTBnPktEvdMA=
-----END RSA PRIVATE KEY-----

People typically redact keys by deleting whole lines, and usually replacing them with [...] and the like. But only about 345 of those 1588 characters (excluding the header and footer) are required to construct the entire key. You can redact about 4/5ths of that giant blob of stuff, and your private parts (or at least, those of your key) are still left uncomfortably exposed.

But Wait! There s More! Remember how I said that everything in the key other than e, p, and q could be derived from those three numbers? Let s talk about one of those numbers: n. This is known as the public modulus (because, along with e, it is also present in the public key). It is very easy to calculate: n = p * q. It is also very early in the key (the second number, in fact). Since n = p * q, it follows that q = n / p. Thus, as long as the key is intact up to p, you can derive q by simple division.

Real World Redaction At this point, I d like to introduce an acquaintance of mine: Mr. Johan Finn. He is the proud owner of the GitHub repo johanfinn/scripts. For a while, his repo contained a script that contained a poorly-redacted private key. He since deleted it, by making a new commit, but of course because git never really deletes anything, it s still available. Of course, Mr. Finn may delete the repo, or force-push a new history without that commit, so here is the redacted private key, with a bit of the surrounding shell script, for our illustrative pleasure:

#Add private key to .ssh folder
cd /home/johan/.ssh/
echo  "-----BEGIN RSA PRIVATE KEY-----
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
 
MIIJKgIBAAKCAgEAxEVih1JGb8gu/Fm4AZh+ZwJw/pjzzliWrg4mICFt1g7SmIE2
TCQMKABdwd11wOFKCPc/UzRH/fHuQcvWrpbOSdqev/zKff9iedKw/YygkMeIRaXB
fYELqvUAOJ8PPfDm70st9GJRhjGgo5+L3cJB2gfgeiDNHzaFvapRSU0oMGQX+kI9
ezsjDAn+0Pp+r3h/u1QpLSH4moRFGF4omNydI+3iTGB98/EzuNhRBHRNq4oBV5SG
Pq/A1bem2ninnoEaQ+OPESxYzDz3Jy9jV0W/6LvtJ844m+XX69H5fqq5dy55z6DW
sGKn78ULPVZPsYH5Y7C+CM6GAn4nYCpau0t52sqsY5epXdeYx4Dc+Wm0CjXrUDEe
Egl4loPKDxJkQqQ/MQiz6Le/UK9vEmnWn1TRXK3ekzNV4NgDfJANBQobOpwt8WVB
rbsC0ON7n680RQnl7PltK9P1AQW5vHsahkoixk/BhcwhkrkZGyDIl9g8Q/Euyoq3
eivKPLz7/rhDE7C1BzFy7v8AjC3w7i9QeHcWOZFAXo5hiDasIAkljDOsdfD4tP5/
wSO6E6pjL3kJ+RH2FCHd7ciQb+IcuXbku64ln8gab4p8jLa/mcMI+V3eWYnZ82Yu
axsa85hAe4wb60cp/rCJo7ihhDTTvGooqtTisOv2nSvCYpcW9qbL6cGjAXECAwEA
AQKCAgEAjz6wnWDP5Y9ts2FrqUZ5ooamnzpUXlpLhrbu3m5ncl4ZF5LfH+QDN0Kl
KvONmHsUhJynC/vROybSJBU4Fu4bms1DJY3C39h/L7g00qhLG7901pgWMpn3QQtU
4P49qpBii20MGhuTsmQQALtV4kB/vTgYfinoawpo67cdYmk8lqzGzzB/HKxZdNTq
s+zOfxRr7PWMo9LyVRuKLjGyYXZJ/coFaobWBi8Y96Rw5NZZRYQQXLIalC/Dhndm
AHckpstEtx2i8f6yxEUOgPvV/gD7Akn92RpqOGW0g/kYpXjGqZQy9PVHGy61sInY
HSkcOspIkJiS6WyJY9JcvJPM6ns4b84GE9qoUlWVF3RWJk1dqYCw5hz4U8LFyxsF
R6WhYiImvjxBLpab55rSqbGkzjI2z+ucDZyl1gqIv9U6qceVsgRyuqdfVN4deU22
LzO5IEDhnGdFqg9KQY7u8zm686Ejs64T1sh0y4GOmGsSg+P6nsqkdlXH8C+Cf03F
lqPFg8WQC7ojl/S8dPmkT5tcJh3BPwIWuvbtVjFOGQc8x0lb+NwK8h2Nsn6LNazS
0H90adh/IyYX4sBMokrpxAi+gMAWiyJHIHLeH2itNKtAQd3qQowbrWNswJSgJzsT
JuJ7uqRKAFkE6nCeAkuj/6KHHMPsfCAffVdyGaWqhoxmPOrnVgECggEBAOrCCwiC
XxwUgjOfOKx68siFJLfHf4vPo42LZOkAQq5aUmcWHbJVXmoxLYSczyAROopY0wd6
Dx8rqnpO7OtZsdJMeBSHbMVKoBZ77hiCQlrljcj12moFaEAButLCdZFsZW4zF/sx
kWIAaPH9vc4MvHHyvyNoB3yQRdevu57X7xGf9UxWuPil/jvdbt9toaraUT6rUBWU
GYPNKaLFsQzKsFWAzp5RGpASkhuiBJ0Qx3cfLyirjrKqTipe3o3gh/5RSHQ6VAhz
gdUG7WszNWk8FDCL6RTWzPOrbUyJo/wz1kblsL3vhV7ldEKFHeEjsDGroW2VUFlS
asAHNvM4/uYcOSECggEBANYH0427qZtLVuL97htXW9kCAT75xbMwgRskAH4nJDlZ
IggDErmzBhtrHgR+9X09iL47jr7dUcrVNPHzK/WXALFSKzXhkG/yAgmt3r14WgJ6
5y7010LlPFrzaNEyO/S4ISuBLt4cinjJsrFpoo0WI8jXeM5ddG6ncxdurKXMymY7
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::.::
:::::::::::::::::::::::::::.::::::::::::::::::::::::::::::::::::
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLlL
 
 
 
YYYYYYYYYYYYYYYYYYYYYyYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
gff0GJCOMZ65pMSy3A3cSAtjlKnb4fWzuHD5CFbusN4WhCT/tNxGNSpzvxd8GIDs
nY7exs9L230oCCpedVgcbayHCbkChEfoPzL1e1jXjgCwCTgt8GjeEFqc1gXNEaUn
O8AJ4VlR8fRszHm6yR0ZUBdY7UJddxQiYOzt0S1RLlECggEAbdcs4mZdqf3OjejJ
06oTPs9NRtAJVZlppSi7pmmAyaNpOuKWMoLPElDAQ3Q7VX26LlExLCZoPOVpdqDH
KbdmBEfTR4e11Pn9vYdu9/i6o10U4hpmf4TYKlqk10g1Sj21l8JATj/7Diey8scO
sAI1iftSg3aBSj8W7rxCxSezrENzuqw5D95a/he1cMUTB6XuravqZK5O4eR0vrxR
AvMzXk5OXrUEALUvt84u6m6XZZ0pq5XZxq74s8p/x1JvTwcpJ3jDKNEixlHfdHEZ
ZIu/xpcwD5gRfVGQamdcWvzGHZYLBFO1y5kAtL8kI9tW7WaouWVLmv99AyxdAaCB
Y5mBAQKCAQEAzU7AnorPzYndlOzkxRFtp6MGsvRBsvvqPLCyUFEXrHNV872O7tdO
GmsMZl+q+TJXw7O54FjJJvqSSS1sk68AGRirHop7VQce8U36BmI2ZX6j2SVAgIkI
9m3btCCt5rfiCatn2+Qg6HECmrCsHw6H0RbwaXS4RZUXD/k4X+sslBitOb7K+Y+N
Bacq6QxxjlIqQdKKPs4P2PNHEAey+kEJJGEQ7bTkNxCZ21kgi1Sc5L8U/IGy0BMC
PvJxssLdaWILyp3Ws8Q4RAoC5c0ZP0W2j+5NSbi3jsDFi0Y6/2GRdY1HAZX4twem
Q0NCedq1JNatP1gsb6bcnVHFDEGsj/35oQKCAQEAgmWMuSrojR/fjJzvke6Wvbox
FRnPk+6YRzuYhAP/YPxSRYyB5at++5Q1qr7QWn7NFozFIVFFT8CBU36ktWQ39MGm
cJ5SGyN9nAbbuWA6e+/u059R7QL+6f64xHRAGyLT3gOb1G0N6h7VqFT25q5Tq0rc
Lf/CvLKoudjv+sQ5GKBPT18+zxmwJ8YUWAsXUyrqoFWY/Tvo5yLxaC0W2gh3+Ppi
EDqe4RRJ3VKuKfZxHn5VLxgtBFN96Gy0+Htm5tiMKOZMYAkHiL+vrVZAX0hIEuRZ
JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
-----END RSA PRIVATE KEY-----" >> id_rsa

Now, if you try to reconstruct this key by removing the obvious garbage lines (the ones that are all repeated characters, some of which aren t even valid base64 characters), it still isn t a key at least, openssl pkey doesn t want anything to do with it. The key is very much still in there, though, as we shall soon see. Using a gem I wrote and a quick bit of Ruby, we can extract a complete private key. The irb session looks something like this:

>> require "derparse"
>> b64 = <<EOF
MIIJKgIBAAKCAgEAxEVih1JGb8gu/Fm4AZh+ZwJw/pjzzliWrg4mICFt1g7SmIE2
TCQMKABdwd11wOFKCPc/UzRH/fHuQcvWrpbOSdqev/zKff9iedKw/YygkMeIRaXB
fYELqvUAOJ8PPfDm70st9GJRhjGgo5+L3cJB2gfgeiDNHzaFvapRSU0oMGQX+kI9
ezsjDAn+0Pp+r3h/u1QpLSH4moRFGF4omNydI+3iTGB98/EzuNhRBHRNq4oBV5SG
Pq/A1bem2ninnoEaQ+OPESxYzDz3Jy9jV0W/6LvtJ844m+XX69H5fqq5dy55z6DW
sGKn78ULPVZPsYH5Y7C+CM6GAn4nYCpau0t52sqsY5epXdeYx4Dc+Wm0CjXrUDEe
Egl4loPKDxJkQqQ/MQiz6Le/UK9vEmnWn1TRXK3ekzNV4NgDfJANBQobOpwt8WVB
rbsC0ON7n680RQnl7PltK9P1AQW5vHsahkoixk/BhcwhkrkZGyDIl9g8Q/Euyoq3
eivKPLz7/rhDE7C1BzFy7v8AjC3w7i9QeHcWOZFAXo5hiDasIAkljDOsdfD4tP5/
wSO6E6pjL3kJ+RH2FCHd7ciQb+IcuXbku64ln8gab4p8jLa/mcMI+V3eWYnZ82Yu
axsa85hAe4wb60cp/rCJo7ihhDTTvGooqtTisOv2nSvCYpcW9qbL6cGjAXECAwEA
AQKCAgEAjz6wnWDP5Y9ts2FrqUZ5ooamnzpUXlpLhrbu3m5ncl4ZF5LfH+QDN0Kl
KvONmHsUhJynC/vROybSJBU4Fu4bms1DJY3C39h/L7g00qhLG7901pgWMpn3QQtU
4P49qpBii20MGhuTsmQQALtV4kB/vTgYfinoawpo67cdYmk8lqzGzzB/HKxZdNTq
s+zOfxRr7PWMo9LyVRuKLjGyYXZJ/coFaobWBi8Y96Rw5NZZRYQQXLIalC/Dhndm
AHckpstEtx2i8f6yxEUOgPvV/gD7Akn92RpqOGW0g/kYpXjGqZQy9PVHGy61sInY
HSkcOspIkJiS6WyJY9JcvJPM6ns4b84GE9qoUlWVF3RWJk1dqYCw5hz4U8LFyxsF
R6WhYiImvjxBLpab55rSqbGkzjI2z+ucDZyl1gqIv9U6qceVsgRyuqdfVN4deU22
LzO5IEDhnGdFqg9KQY7u8zm686Ejs64T1sh0y4GOmGsSg+P6nsqkdlXH8C+Cf03F
lqPFg8WQC7ojl/S8dPmkT5tcJh3BPwIWuvbtVjFOGQc8x0lb+NwK8h2Nsn6LNazS
0H90adh/IyYX4sBMokrpxAi+gMAWiyJHIHLeH2itNKtAQd3qQowbrWNswJSgJzsT
JuJ7uqRKAFkE6nCeAkuj/6KHHMPsfCAffVdyGaWqhoxmPOrnVgECggEBAOrCCwiC
XxwUgjOfOKx68siFJLfHf4vPo42LZOkAQq5aUmcWHbJVXmoxLYSczyAROopY0wd6
Dx8rqnpO7OtZsdJMeBSHbMVKoBZ77hiCQlrljcj12moFaEAButLCdZFsZW4zF/sx
kWIAaPH9vc4MvHHyvyNoB3yQRdevu57X7xGf9UxWuPil/jvdbt9toaraUT6rUBWU
GYPNKaLFsQzKsFWAzp5RGpASkhuiBJ0Qx3cfLyirjrKqTipe3o3gh/5RSHQ6VAhz
gdUG7WszNWk8FDCL6RTWzPOrbUyJo/wz1kblsL3vhV7ldEKFHeEjsDGroW2VUFlS
asAHNvM4/uYcOSECggEBANYH0427qZtLVuL97htXW9kCAT75xbMwgRskAH4nJDlZ
IggDErmzBhtrHgR+9X09iL47jr7dUcrVNPHzK/WXALFSKzXhkG/yAgmt3r14WgJ6
5y7010LlPFrzaNEyO/S4ISuBLt4cinjJsrFpoo0WI8jXeM5ddG6ncxdurKXMymY7
EOF
>> b64 += <<EOF
gff0GJCOMZ65pMSy3A3cSAtjlKnb4fWzuHD5CFbusN4WhCT/tNxGNSpzvxd8GIDs
nY7exs9L230oCCpedVgcbayHCbkChEfoPzL1e1jXjgCwCTgt8GjeEFqc1gXNEaUn
O8AJ4VlR8fRszHm6yR0ZUBdY7UJddxQiYOzt0S1RLlECggEAbdcs4mZdqf3OjejJ
06oTPs9NRtAJVZlppSi7pmmAyaNpOuKWMoLPElDAQ3Q7VX26LlExLCZoPOVpdqDH
KbdmBEfTR4e11Pn9vYdu9/i6o10U4hpmf4TYKlqk10g1Sj21l8JATj/7Diey8scO
sAI1iftSg3aBSj8W7rxCxSezrENzuqw5D95a/he1cMUTB6XuravqZK5O4eR0vrxR
AvMzXk5OXrUEALUvt84u6m6XZZ0pq5XZxq74s8p/x1JvTwcpJ3jDKNEixlHfdHEZ
ZIu/xpcwD5gRfVGQamdcWvzGHZYLBFO1y5kAtL8kI9tW7WaouWVLmv99AyxdAaCB
Y5mBAQKCAQEAzU7AnorPzYndlOzkxRFtp6MGsvRBsvvqPLCyUFEXrHNV872O7tdO
GmsMZl+q+TJXw7O54FjJJvqSSS1sk68AGRirHop7VQce8U36BmI2ZX6j2SVAgIkI
9m3btCCt5rfiCatn2+Qg6HECmrCsHw6H0RbwaXS4RZUXD/k4X+sslBitOb7K+Y+N
Bacq6QxxjlIqQdKKPs4P2PNHEAey+kEJJGEQ7bTkNxCZ21kgi1Sc5L8U/IGy0BMC
PvJxssLdaWILyp3Ws8Q4RAoC5c0ZP0W2j+5NSbi3jsDFi0Y6/2GRdY1HAZX4twem
Q0NCedq1JNatP1gsb6bcnVHFDEGsj/35oQKCAQEAgmWMuSrojR/fjJzvke6Wvbox
FRnPk+6YRzuYhAP/YPxSRYyB5at++5Q1qr7QWn7NFozFIVFFT8CBU36ktWQ39MGm
cJ5SGyN9nAbbuWA6e+/u059R7QL+6f64xHRAGyLT3gOb1G0N6h7VqFT25q5Tq0rc
Lf/CvLKoudjv+sQ5GKBPT18+zxmwJ8YUWAsXUyrqoFWY/Tvo5yLxaC0W2gh3+Ppi
EDqe4RRJ3VKuKfZxHn5VLxgtBFN96Gy0+Htm5tiMKOZMYAkHiL+vrVZAX0hIEuRZ
EOF
>> der = b64.unpack("m").first
>> c = DerParse.new(der).first_node.first_child
>> version = c.value
=> 0
>> c = c.next_node
>> n = c.value
=> 80071596234464993385068908004931... # (etc)
>> c = c.next_node
>> e = c.value
=> 65537
>> c = c.next_node
>> d = c.value
=> 58438813486895877116761996105770... # (etc)
>> c = c.next_node
>> p = c.value
=> 29635449580247160226960937109864... # (etc)
>> c = c.next_node
>> q = c.value
=> 27018856595256414771163410576410... # (etc)

What I ve done, in case you don t speak Ruby, is take the two chunks of plausible-looking base64 data, chuck them together into a variable named b64, unbase64 it into a variable named der, pass that into a new DerParse instance, and then walk the DER value tree until I got all the values I need. Interestingly, the q value actually traverses the split in the two chunks, which means that there s always the possibility that there are lines missing from the key. However, since p and q are supposed to be prime, we can sanity check them to see if corruption is likely to have occurred:

>> require "openssl"
>> OpenSSL::BN.new(p).prime?
=> true
>> OpenSSL::BN.new(q).prime?
=> true

Excellent! The chances of a corrupted file producing valid-but-incorrect prime numbers isn t huge, so we can be fairly confident that we ve got the real p and q. Now, with the help of another one of my creations we can use e, p, and q to create a fully-operational battle key:

>> require "openssl/pkey/rsa"
>> k = OpenSSL::PKey::RSA.from_factors(p, q, e)
=> #<OpenSSL::PKey::RSA:0x0000559d5903cd38>
>> k.valid?
=> true
>> k.verify(OpenSSL::Digest::SHA256.new, k.sign(OpenSSL::Digest::SHA256.new, "bob"), "bob")
=> true

and there you have it. One fairly redacted-looking private key brought back to life by maths and far too much free time. Sorry Mr. Finn, I hope you re not still using that key on anything Internet-facing.

What About Other Key Types? EC keys are very different beasts, but they have much the same problems as RSA keys. A typical EC key contains both private and public data, and the public portion is twice the size so only about 1/3 of the data in the key is private material. It is quite plausible that you can redact an EC key and leave all the actually private bits exposed.

What Do We Do About It? In short: don t ever try and redact real private keys. For documentation purposes, just put KEY GOES HERE in the appropriate spot, or something like that. Store your secrets somewhere that isn t a public (or even private!) git repo. Generating a dummy private key and sticking it in there isn t a great idea, for different reasons: people have this odd habit of reusing demo keys in real life. There s no need to encourage that sort of thing.

---

1. Technically the pieces aren t 100% aligned with the underlying DER, because of how base64 works. I felt it was easier to understand if I stuck to chopping up the base64, rather than decoding into DER and then chopping up the DER.

Review: Gideon the Ninth, by Tamsyn Muir

Series:	The Locked Tomb #1
Publisher:	Tor
Copyright:	September 2019
ISBN:	1-250-31317-1
Format:	Kindle
Pages:	448

Despite being raised there, Gideon Nav is an outsider in the Ninth House. Her mother, already dead, fell from the sky with a one-day-old Gideon in tow, leaving her an indentured servant. She's a grumpy, caustic teenager in a world of moldering corpses, animated skeletons, and mostly-dead adults whose parts are falling off. Her world is sword fighting, dirty magazines, a feud with the house heir Harrowhark, and a determination to escape the terms of her indenture. Gideon does get off the planet, but not the way that she expects. She doesn't get accepted into the military. She ends up in the middle of a bizarre test, or possibly an ascension rite, mingling with and competing with the nobility of the empire alongside her worst enemy. I struggled to enjoy the beginning of Gideon the Ninth. Gideon tries to carry the story on pure snark, but it is very, very goth. If you like desiccated crypts, mostly-dead goons, betrayal, frustration, necromancers, black robes, disturbing family relationships, gloom, and bitter despair, the first six chapters certainly deliver, but I was sick of it by the time Gideon gets out. Thankfully, the opening is largely unlike the rest of the book. What starts as an over-the-top teenage goth rebellion turns into a cross between a manor house murder mystery and a competitive escape room. This book is a bit of a mess, but it's a glorious mess. It's also the sort of glorious mess that I don't think would have been written or published twenty years ago, and I have a pet theory that attributes this to the invigorating influence of fanfic and writers who grew up reading and writing it. I read a lot of classic science fiction and epic fantasy as a teenager. Those books have many merits, obviously, but emotional range is not one of them. There are a few exceptions, but on average the genre either focused on puzzles and problem solving (how do we fix the starship, how do we use the magic system to take down the dark god) or on the typical "heroic" (and male-coded) emotions of loyalty, bravery, responsibility, authority, and defiance of evil. Characters didn't have messy breakups, frenemies, anxiety, socially-awkward love affairs, impostor syndrome, self-hatred, or depression. And authors weren't allowed to fall in love with the messiness of their characters, at least on the page. I'm not enough of a scholar to make the argument well, but I suspect there's a case to be made that fanfic exists partially to fill this gap. So much of fanfic starts from taking the characters on the canonical page or screen and letting them feel more, live more, love more, screw up more, and otherwise experience a far wider range of human drama, particularly compared to what made it into television, which was even more censored than what made it into print. Some of those readers and writers are now writing for publication, and others have gone into publishing. The result, in my theory, is that the range of stories that are acceptable in the genre has broadened, and the emotional texture of those stories has deepened. Whether or not this theory is correct, there are now more novels like this in the world, novels full of grudges, deflective banter, squabbling, messy emotional processing, and moments of glorious emotional catharsis. This makes me very happy. To describe the emotional payoff of this book in any more detail would be a huge spoiler; suffice it to say that I unabashedly love fragile competence and unexpected emotional support, and adore this book for containing it. Gideon's voice, irreverent banter, stubborn defiance, and impulsive good-heartedness are the center of this book. At the start, it's not clear whether there will be another likable character in the book. There will be, several of them, but it takes a while for Gideon to find them or for them to become likable. You'll need to like Gideon well enough to stick with her for that journey. I read books primarily for the characters, not for the setting, and Gideon the Ninth struck some specific notes that I will happily read endlessly. If that doesn't match your preferences, I would not be too surprised to hear you bounced off the book. There's a lot here that won't be to everyone's taste. The setting felt very close to Warhammer 40K: an undead emperor that everyone worships, endless war, necromancy, and gothic grimdark. The stage for most of the book is at least more light-filled, complex, and interesting than the Ninth House section at the start, but everything is crumbling, drowning, broken, or decaying. There's quite a lot of body horror, grotesque monsters, and bloody fights. And the ending is not the best part of the book; roughly the last 15% of the novel is composed of two running fight scenes against a few practically unkillable and frankly not very interesting villains. I got exhausted by the fighting long before it was over, and the conclusion is essentially a series cliffhanger. There are also a few too many characters. The collection of characters and the interplay between the houses is one of the strengths of this book, but Muir sets up her story in a way that requires eighteen significant characters and makes the reader want to keep track of all of them. It took me about halfway through the book before I felt like I had my bearings and wasn't confusing one character for another or forgetting a whole group of characters. That said, most of the characters are great, and the story gains a lot from the interplay of their different approaches and mindsets. Palamedes Sextus's logical geekery, in particular, is a great counterpoint to the approaches of most of the other characters. The other interesting thing Muir does in this novel that I've not seen before, and that feels very modern, is to set the book in essentially an escape room. Locking a bunch of characters in a sprawling mansion until people start dying is an old fictional trope, but this one has puzzles, rewards, and a progressive physical structure that provides a lot of opportunities to motivate the characters and give them space to take wildly different problem-solving approaches. I liked this a lot, and I'm looking forward to seeing it in future books. This is not the best book I've read, but I thoroughly enjoyed it, despite some problems with the ending. I've already pre-ordered the sequel. Followed by Harrow the Ninth. Rating: 8 out of 10

Rolling Stone has an insightful article about why the Christian Right supports Trump and won t stop supporting him no matter what he does [1]. Interesting article about Data Oriented Architecture [2]. Quarantine Will normalise WFH and Recession will Denormalise Jobs [3]. I guess we can always hope that after a disaster we can learn to do things better than before. Tyre wear is worse than exhaust for small particulate matter [4]. We need better tyres and legal controls over such things. Scott Santens wrote an insightful article about the need for democracy and unconditional basic income [5]. In ancient Greece, work was regarded as a curse is an extreme position but strongly supported by evidence. In his essay In Praise of Idleness, Bertrand Russell wrote Modern methods of production have given us the possibility of ease and security for all; we have chosen, instead, to have overwork for some and starvation for others. Hitherto we have continued to be as energetic as we were before there were machines; in this we have been foolish, but there is no reason to go on being foolish forever. Cory Doctorow wrote an insightful article for Locus titled A Lever Without a Fulcrum Is Just a Stick about expansions to copyright laws [6]. One of his analogies is that giving a bullied kid more lunch money just allows the bullies to steal more money, with artists being bullied kids and lunch money being the rights that are granted under copyright law. The proposed solution includes changes to labor and contract law, presumably Cory will write other articles in future giving the details of his ideas in this regard. The Register has an amusing article about the trial of a former CIA employee on trial for being the alleged vault 7 leaker [7]. Both the prosecution and the defence are building their cases around the defendent being a jerk. The article exposes poor security and poor hiring practices in the CIA. CNN has an informative article about Finland s war on fake news [8]. As Finland has long standing disputes with Russia they have had more practice at dealing with fake news than most countries. The Times of Israel has an interesting article about how the UK used German Jews to spy on German prisoners of war [9]. Cory Doctorow wrote an insightful article Data is the New Toxic Waste about how collecting personal data isn t an asset, it s a liability [10]. Ulrike Uhlig wrote an insightful article about Control Freaks , analysing the different meanings of control, both positive and negative [11]. 538 has an informative article about the value of statistical life [12]. It s about $9M per person in the US, which means a mind-boggling amount of money should be spent to save the millions of lives that will be potentially lost in a natural disaster (like Coronavirus). NPR has an interesting interview about Crypto AG, the Swiss crypto company owned by the CIA [13]. I first learned of this years ago, it s not new, but I still learned a lot from this interview.

• [1] http://tinyurl.com/r2zyrpz
• [2] https://blog.eyas.sh/2020/03/data-oriented-architecture/
• [3] https://abe-winter.github.io/2020/03/07/wfhdemic.html
• [4] http://tinyurl.com/tf4yj4m
• [5] http://tinyurl.com/y9dcnzma
• [6] http://tinyurl.com/v3mxgh5
• [7] https://www.theregister.co.uk/2020/03/05/cia_leak_trial/
• [8] http://tinyurl.com/y6cn42us
• [9] http://tinyurl.com/y67mo4rr
• [10] http://tinyurl.com/wucsgw8
• [11] https://curlybracket.net/2020/03/16/control-freaks.html
• [12] http://tinyurl.com/s6e7luk
• [13] http://tinyurl.com/qpx3dah

Narabu is a new intraframe video codec. You probably want to read part 1, part 2, part 3, part 4 and part 5 first. Like I wrote in part 5, there basically isn't a big splashy ending where everything is resolved here; you're basically getting some graphs with some open questions and some interesting observations. First of all, though, I'll need to make a correction: In the last part, I wrote that encoding takes 1.2 ms for 720p luma-only on my GTX 950, which isn't correct I remembered the wrong number. The right number is 2.3 ms, which I guess explains even more why I don't think it's acceptable at the current stage. (I'm also pretty sure it's possible to rearchitect the encoder so that it's much better, but I am moving on to other video-related things for the time being.) I encoded a picture straight off my DSLR (luma-only) at various resolutions, keeping the aspect. Then I decoded it a bunch of times on my GTX 950 (low-end last-generation NVIDIA) and on my HD 4400 (ultraportable Haswell laptop) and measured the times. They're normalized for megapixels per second decoded; remember that doubling width (x axis) means quadruple the pixels. Here it is: I'm not going to comment much beyond two observations:

• Caches matter, even on GPU. This is the same data over and over again (so small images get an unrealistic boost), so up to a certain point, it's basically all in L1.
• The GTX 950 doesn't really run away from the Intel card before it's getting enough data to chew on. Bigger GPUs don't have faster cores they're just more parallel.

Encoding only contains the GTX 950 because I didn't finish the work to get that single int64 divide off: This is interesting. I have few explanations. Probably more benchmarking and profiling would be needed to make sense of any of it. In fact, it's so strange that I would suspect a bug, but it does indeed seem to create a valid bitstream that is decoded by the decoder. Do note, however, that seemingly even on the smallest resolutions, there's a 1.7 ms base cost (you can't see it on the picture, but you'd see it in an unnormalized graph). I don't have a very good explanation for this either (even though there are some costs that are dependent on the alphabet size instead of the number of pixels), but figuring it out would probably be a great start for getting the performance up. So that concludes the series, on a cliffhanger. :-) Even though it's not in a situation where you can just take it and put it into something useful, I hope it was an interesting introduction to the GPU! And in the meantime, I've released version 1.6.3 of Nageru, my live video mixer (also heavily GPU-based) with various small adjustments and bug fixes found before and during Tr ndisk. And Movit is getting compute shaders for that extra speed boost, although parts of it is bending my head. Exciting times in GPU land :-)

Review: Waypoint Kangaroo, by Curtis C. Chen

Series:	Kangaroo #1
Publisher:	Thomas Dunne
Copyright:	June 2016
ISBN:	1-250-08179-3
Format:	Kindle
Pages:	312

Disclaimer: Curtis was a classmate of mine at Stanford and part of the same social circle. That was a surprisingly long time ago. Kangaroo is a spy (and, for this book, you should think James Bond). Agency training, fake identities, lots of gadgets, grumpy yet ridiculously competent support staff... the typical package. But Kangaroo also has a special power, which is the entire reason he ended up in the position he has. He's apparently the only person in the world who can open the pocket: a hole into another dimension, which can function as infinite storage and quite a bit more. Waypoint Kangaroo opens with the tail end of a mission and Kangaroo in action, as an introduction to Kangaroo's first-person narrative voice, job, and the capabilities of the pocket. But the real story starts when Kangaroo is sent on vacation. The office is being audited, Kangaroo hasn't had time off basically ever, and his boss insists on a trip to Mars on the space equivalent of a cruise ship. No work. An expense account. Just relax and have fun. Kangaroo isn't sure he knows how to not work. Or how to avoid boredom when trying hard to not work. It leads to probably ill-advised decisions like falling in love at first glance with the chief engineer, or going on entirely unauthorized spacewalks in the middle of the night. It's very lucky for him that the captain of this commercial cruise ship appears to also work for his agency. And it's good for his inability to stop working that there's a murder on board. For a first novel, this is refreshingly free of a lot of first novel problems. It's lean, well-structured, easy to follow, moves right along, and doesn't feel over-stuffed with exposition or world-building. There's an interplanetary war in the past background, and of course a lot of loving description of the precise mechanics of the pocket and the tricks with momentum and retrieval Kangaroo can do with it, but the book never falls into too much explanation. And the plot is satisfyingly twisty. It's an action story plot, to be clear: don't expect deep puzzles or complex deduction. But there are enough players and hidden motives to keep things interesting. The downside is that I didn't like Kangaroo very much. He's a bit of an ass. Some of this goes with the spy novel territory, and some of it is good (if occasionally grating) characterization. Kangaroo doesn't know how to turn off the part of his brain that makes everything a mission. But his flippant, know-it-all attitude got on my nerves after a book full of first-person narration, and while (full credit to Curtis here) the romance in this book is clearly consensual and stays well away from the creepy romances so common in spy stories, the love-at-first sight bits and some of Kangaroo's awkward reactions provoked more eye-rolling than enjoyment. A lot of this is just personal taste, but that's the peril of books told with first-person narration. The reader has to really like the protagonist to spend a whole book in their head. If that relationship doesn't click, the supporting characters have a harder time salvaging the experience. Waypoint Kangaroo avoids the problem of too many loving descriptions of guns, partly because it's a spy novel and instead has loving descriptions of spy equipment in a future that supports implanted devices. I think there was a smidgen too much of this, but it was within genre conventions and spy stuff is more interesting than guns. But (and I admit that this is probably idiosyncratic), it also had way too many loving descriptions of alcohol and one drunk scene. I don't care to ever read another book with a drunk protagonist (particularly first-person), and I care considerably less about alcohol than I do about spy equipment or guns. That said, I still liked this well enough that I'll probably buy the sequel. (No cliffhangers; Waypoint Kangaroo is a complete story. But this is a character who could easily support a long episodic series.) The pocket is a neat gimmick, the world background is at least mildly interesting, and some of the supporting characters were excellent. (Particularly the security chief and the engineer.) I might even warm to Kangaroo over time if subsequent stories stay more on his creative fast-talking rather than his drinking and awkward romances. I don't think this is quite good enough for me to recommend it, but if you're in the mood for a light and fast-moving first-person Bond-style story with science fiction trappings, it does deliver. Rating: 6 out of 10

Last month, there has been an interesting discussion about off-line GnuPG keys and their storage systems on the debian-project@l.d.o mailing list. I tried to summarise it in the Debian wiki, in particular by creating two new pages.

Review: Regeneration, by Julie E. Czerneda

Series:	Species Imperative #3
Publisher:	DAW
Copyright:	2006
ISBN:	0-7564-0345-6
Format:	Hardcover
Pages:	543

This is the third book of the Species Imperative trilogy, and this is the type of trilogy that's telling a single story in three books. You don't want to read this out of order, and I'll have to be cautious about aspects of the plot to not spoil the earlier books. Mac is still recovering from the effects of the first two books of the series, but she's primarily worried about a deeply injured friend. Worse, that friend is struggling to explain or process what's happened, and the gaps in her memory and her very ability to explain may point at frightening, lingering risks to humanity. As much as she wants to, Mac can't give her friend all of her focus, since she's also integral to the team trying to understand the broader implications of the events of Migration. Worse, some of the non-human species have their own contrary interpretations that, if acted on, Mac believes would be desperately risky for humanity and all the other species reachable through the transects. That set of competing priorities and motivations eventually sort themselves out into a tense and rewarding multi-species story, but they get off to an awkward start. The first 150 pages of Regeneration are long on worry, uncertainty, dread, and cryptic conversations, and short on enjoyable reading. Czerneda's recaps of the previous books are appreciated, but they weren't very smoothly integrated into the story. (I renew my occasional request for series authors to include a simple plot summary of the previous books as a prefix, without trying to weave it into the fiction.) I was looking forward to this book after the excellent previous volumes, but struggled to get into the story. That does change. It takes a bit too long, with a bit too much nameless dread, a bit too much of an irritating subplot between Fourteen and Oversight that I didn't think added anything to the book, and not enough of Mac barreling forward doing sensible things. But once Mac gets back into space, with a destination and a job and a collection of suspicious (or arrogant) humans and almost-incomprehensible aliens to juggle, Czerneda hits her stride. Czerneda doesn't entirely avoid Planet of the Hats problems with her aliens, but I think she does better than most of science fiction. Alien species in this series do tend to be a bit all of a type, and Mac does figure them out by drawing conclusions from biology, but those conclusions are unobvious and based on Mac's mix of biological and human social intuition. They refreshingly aren't as simple as biology completely shaping culture. (Czerneda's touch is more subtle than James White's Sector General, for example.) And Mac has a practical, determined, and selfless approach that's deeply likable and admirable. It's fun as a reader to watch her win people over by just being competent, thoughtful, observant, and unrelentingly ethical. But the best part of this book, by far, are the Sinzi. They first appeared in the second book, Migration, and seemed to follow the common SF trope of a wise elder alien race that can bring some order to the universe and that humanity can learn from. They, or more precisely the one Sinzi who appeared in Migration, was very good at that role. But Czerneda had something far more interesting planned, and in Regeneration they become truly alien in their own right, with their own nearly incomprehensible way of viewing the universe. There are so many ways that this twist can go wrong, and Czerneda avoids all of them. She doesn't undermine their gravitas, nor does she elevate them to the level of Arisians or other semi-angelic wise mentors of other series. Czerneda makes them different in profound ways that are both advantage and disadvantage, pulls that difference into the plot as a complicating element, and has Mac stumble yet again into a role that is accidentally far more influential than she intends. Mac is the perfect character to do that to: she has just the right mix of embarrassment, ethics, seat-of-the-pants blunt negotiation skills, and a strong moral compass. Given a lever and a place to stand, one can believe that Mac can move the world, and the Sinzi are an absolutely fascinating lever. There are also three separate, highly differentiated Sinzi in this story, with different goals, life experience, personalities, and levels of gravitas. Czerneda's aliens are good in general, but her focus is usually more on biology than individual differentiation. The Sinzi here combine the best of both types of character building. I think the ending of Regeneration didn't entirely work. After all the intense effort the characters put into understanding the complexity of the universe over the course of the series, the denouement has a mopping-up feel and a moral clarity that felt a bit too easy. But the climax has everything I was hoping for, there's a lot more of Mac being Mac, and I loved every moment of the Sinzi twist. Now I want a whole new series exploring the implications of the Sinzi's view of the universe on the whole history of galactic politics that sat underneath this story. But I'll settle for moments of revelation that sent shivers down my spine. This is a bit of an uneven book that falls short of its potential, but I'll remember it for a long time. Add it on to a deeply rewarding series, and I will recommend the whole package unreservedly. The Species Imperative is excellent science fiction that should be better-known than it is. I still think the romance subplot was unfortunate, and occasionally the aliens get too cartoony (Fourteen, in particular, goes a bit too far in that direction), but Czerneda never lingers too long on those elements. And the whole work is some of the best writing about working scientific research and small-group politics that I've read. Highly recommended, but read the whole series in order. Rating: 9 out of 10

Debconf17 has come and gone by too fast, so we all could use a moment looing back at all the fun and serious happenings of the main event in the Debian social calendar. You can find my full photo gallery on Google, Flickr and Debconf Share. Make sure to check out the Debconf17 group photo and as an extra special treat for you - enjoy the "living" Debconf17 group photo!

Six month ago, I wrote a lengthy post about my new computer hurting my eyes. I haven't made any progress with that, but I've accidentally upgraded my work computer from Kernel 4.4 to 4.8 and the nvidia drivers from 340.96-4 to 375.66-2. Well, my work computer now hurts, I've switched back to the previous kernel and drivers, I hope it'll be back to normal.Any ideas of something specific that changed, either between 4.4 and 4.8 (kernel startup code, default framebuffer modes, etc ?), or between the 340.96 and the 375.66 drivers ? In any case, I'll try that specific combination of kernels/drivers home to see if I can get it to a useable state.Update, July 23rdI reverted to the earlier drivers/kernel, to no avail. But it seems in fact the problem with my work computer is linked to an allergy of some kind, since antihistamine drugs have an effect (there are construction works in my lab, perhaps they are the cause ?). No idea still for my home computer, for which the problem is definitely not solved.

There are quite some events and dates ahead for hackers, makers, debianers and hackerfunk listeners: Crowdfunding for ZeTeCo Camp in July Ends in Two Days! You might have heard of the ZeTeCo Camp near Schaffhausen in July. If you want ot come or at least support that event, please contribute to their crowdfunding campaign. They have more than 90% of their goal funded and there s less only about two days left to reach their funding goal. If it doesn t get funded in time, the event may be be on a knife edge. Debian Bug Squashing Party in Zurich this Weekend One week before the More-than-a-BSP at the Mozilla office in Paris there will also be a Debian Bug Squashing Party (BSP) in Z rich at the CCCZH Hackerspace R schtibach . We ll start on Friday, the 5th of May 2017 in the late afternoon, probably around 4pm or 5pm, and will end on Sunday, the 7th of May 2017 also in the late afternoon. Please add yourself to the according section on the BSP s wiki page if you want to join us to squash the hopefully not that many left over bugs in testing. Unfortunately we didn t notice two date clashes when we set the date for the BSP during the Annual General Meeting (AGM) of the Debian.ch Association earlier this year: Linux Presentation Day this Saturday, 6th of May 2017 On the one hand there will be the Swiss Edition of the Linux Presentation Day (LPD) on Saturday, the 6th of May 2017 including the LPD in Z rich. The latter will take place in the same building as the BSP, just on a different floor: The BSP will be at floor B3 in the CCCZH Hackerspace and the LPD will be on the ground floor at Revamp-IT in the former ZKB foyer. 10 Years Hackerfunk: Show on 6th, Party on 13th of May 2017 And on the other hand, Venty s and my (German dialect) radio show and podcast Hackerfunk will have it s 10th anniversary show also on that Saturday. So I ll vanish from BSP for a few hours on Saturday evening for broadcasting this very special Hackerfunk episode on Radio Radius. But since Venty and me didn t want to make yet another big event at R schtibach on the same weekend, we ll do the Hackerfunk 10th Anniversary Party one weekend later on Saturday the 13th of May 2017 also at the CCCZH Hackerspace R schtibach . A separate announcement on https://www.hackerfunk.ch/ (also in the RSS feed there) will follow.

Linux usage in networking hardware has been on the rise for some time. During the latest Netdev conference held in Montreal this April, people talked seriously about Linux running on high end, "top of rack" (TOR) networking equipment. Those devices have long been the realm of proprietary hardware and software companies like Cisco or Juniper, but Linux seems to be making some significant headway into the domain. According to Shrijeet Mukherjee, VP of Engineering at Cumulus Networks: "we are seeing a 28% adoption rate in the Fortune 50" companies. As someone who has worked in system administration and networking for over a decade, I was surprised by this: switches have always been black boxes of proprietary hardware that we barely get a shell into. But as more and more TOR hardware gets Linux support, some of that work trickles down outside of that niche. Are we really seeing the rise of Linux in high-end networking hardware?

Linux as the standard interface During his keynote at Netdev, Jesse Brandeburg explained that traffic is exploding on the Internet: "From 2006 to 2016 the compound annual growth rate was 78% of network traffic. So the network traffic is growing like crazy. In 2010 to 2023, it's going to grow by a thousand times." He also mentioned Intel was working on devices that could do up to 400 Gbps. In his talk, he argued that Linux has a critical place in the modern world by repeating the mantra that "everything is on the network, and the network runs Linux", but does it really? Through Android, Linux has become the most popular end-user device operating system and is also dominant on the server but what about the actual fabric of the network: the routers and switches that interconnect all those machines? Mukherjee, in his own keynote, argued that even though Linux has achieved dominance in the server and virtualization markets, there is a "discontinuity [...] at the boundary where the host and the network meet" and argued "that the host without the network will not survive". In other words, proprietary hardware and software in the network threaten free software in the server. Even though some manufacturers are already providing a "Linux interface" in their hardware, it is often only some sort of compatibility shell which might be compared with the Ubuntu compatibility layer in Windows: it's not a real Linux. Mukherjee pushed this idea further by saying that those companies are limiting themselves by not using the full Linux stack. He presented Linux as the "top vehicle for innovation" that provides a featureful network stack, citing VXLAN, eBPF, and Quagga as prime features used on switches. Mukherjee also praised the diversity of the user-space Linux ecosystem as something that commercial alternatives can't rival; he compared the faster Linux development to the commercial sector where similar top features stay in the beta stage for up to 3 years. Because of its dominance in the server market, consumers are expecting a Linux-like interface on their networking gear now, which means Linux could be the standard interface all providers strive toward. As a Debian developer, I can't help but smile at the thought; if there's one thing we have not been able to do among Linux distributions, it's pretty much standardize user space in a consistent interface. POSIX is old and incomplete, the FHS is showing its age, and most distributions have abandoned LSB. Yet, the idea is certainly enticing: there is a base set of tools and applications, especially for the Linux kernel, that are standardized: iproute2, ethtool, and iptables are generally consistent across distributions, even though each distribution has its own way of using them. Yet Linux is not dominant, why? Mukherjee identified the problem as "packaging issues" and listed a set of features he would like Linux to improve:

• Standardization of the ethtool interface. The idea is to make ethtool a de-facto standard to manage switches and ports. Mukherjee gave the example that data centers spend more money on cables than any other hardware and explained that making it easier to identify cables is therefore a key functionality. Getting consistent interface naming was also a key problem identified by numerous participants at the conference. While systemd tried to fix this with the predictable network interface names specification, the interface names are not necessarily predictable across virtual machines or on special hardware; in fact, this was the topic of the first talk of the conference. ethtool also needs to support interfaces that run faster than 1 Gbps, something that still has limited support in Linux at the moment.
• Scaling of the Linux bridge. Through the rise of "software defined networking", we are likely to see multi-switch virtual environments that need to scale to hundreds of interfaces and devices easily. This is something the Linux bridge was never designed to do and it's showing scalability issues. During the conference, there was hope that the new XDP and eBPF developments could help, but also concerns this would create yet another bridge layer inside the kernel.

Cumulus's goal seems to be to establish Linux as the industry standard for this new era of networking and it is not alone. Through its Open Compute project, Facebook is sharing open designs of data center products and, while we have yet to see commercial off-the-shelf (COTS) 24 and 48-port gigabit switches trickle down to the consumer market, the company is definitely deploying new hardware based on those specifications in its own data centers, and those devices are often based on Linux.

The Linux switch implementation So how exactly do switches work in Linux? The Linux kernel manipulates switches with three different operation structures: switchdev_ops, which we previously covered, ethtool_ops, and netdev_ops. Certain switches, however, also need distributed switch architecture (DSA) features to be properly handled. DSA is a more obscure part of the network stack that allows Linux to represent hardware switches or chains of switches using regular Linux tools like bridge, ifconfig, and so on. While switchdev is a new layer, DSA has been in the kernel since 2.6.28 in 2008. Originally developed to support Marvell switches, DSA is now a generic layer deployed in WiFi access points, set-top boxes, on-board flight entertainment systems, trains, and other industrial equipment. Switches that have an Ethernet controller need DSA, whereas the kernel can support switches without Ethernet controllers directly with switchdev drivers. The first years of DSA's development consisted only of basic maintenance but, in the last three years, DSA has seen a resurgence of contributions, as part of Linux networking push to support hardware offloading and network switches. Between 2014 and 2015, DSA added support for Broadcom hardware, wake on LAN, and hardware port bridging, among other features. DSA's development was parallel to swconfig, written by the OpenWrt project to support the small office and home office (SOHO) routers that the project is targeting. The main difference between swconfig and DSA is that DSA-supported switches show one network interface per port, whereas swconfig-configured switches show up as a single port, which limits the amount of information that can be extracted from the switch. For example, you cannot have per-port traffic statistics with swconfig. That limitation is what led to the creation of the switchdev framework, when swconfig was proposed (then refused) for inclusion in mainline. Another goal of switchdev was to support bridge hardware offloading and network interface card (NIC) virtualization. Also, whereas swconfig uses virtual LAN (VLAN) tagging to address ports, DSA enables the use of device-specific tagging headers to address different ports, which enables DSA to have better control over the switches. This allows, for example, DSA to do internet group management protocol (IGMP) snooping or implement the spanning tree protocol, whereas swconfig doesn't have those features. Some switches are actually connected to the host CPU through an Ethernet interface instead of regular PCI-Express interface, and DSA supports this as well. One advantage that remains in the swconfig approach is that it treats the internal switch as a simple external switch, and addresses ports with standard VLAN tags. This is something DSA could do, as well, but no one has bothered implementing this just yet. For now, DSA drivers use device-specific tagging mechanisms that limit the number of supported devices. Other areas of future improvement for DSA are better multi-chip support, IGMP snooping, and bonding, as well as firewall, NAT, and TC offloading.

Where is the freedom? Given all those technical improvements, you might rightfully wonder if your own wireless router or data center switch runs Linux. In recent years, we have seen more and more networking devices shipped with Linux and sometimes even OpenWrt (e.g. in the case of the Turris Omnia, which we previously covered), and especially on SOHO routers, but it sometimes means a crippled operating system that only offers you a proprietary web interface. But at least those efforts make it easier to deploy free operating systems on those devices. Based on my experience running OpenWrt on wireless routers to build the Montreal mesh network, deploying Linux on routers and switches has always been a manual process. The Ubiquiti hardware being used in the mesh network comes with an OpenWrt derivative, but it includes proprietary drivers and a proprietary web interface. To use the mesh networking protocol that was chosen, it was necessary to deploy custom OpenWrt images by hand. For years, it was a continuous struggle for OpenWrt developers to liberate generation after generation of proprietary hardware with companies like Cisco locking down the venerable WRT platform in 2006 and the US Federal Communications Commission (FCC) rules that forced TP-Link to block free software on its routers, a change that was later reverted. Most hardware providers are obviously not dedicated to software freedom: deploying Linux on their hardware is for them an economic, not political choice. As you might expect, a "Linux router" these days often means a closed device and operating system, using Linux as the only free component. I had the privilege of doing some reverse engineering on the SmartRG SR603n VDSL modem, which also doubles as a WiFi router and VoIP phone adapter. I was able to get a shell on that machine, and what I found was a custom operating system built on top of the Linux kernel. I wrote a detailed report about this two years ago and the situation then was pretty grim. Another experience I had was working over a decade in data centers, which tells an even worse story: most switches and routers there are not running free software at all. We have deployed HP ProCurve switches that provide free (as in beer) software updates and have struggled for years to find free (as in speech) software alternatives for those. We built our own routers using COTS server hardware, at a significant performance cost over the dedicated application-specific integrated circuits (ASICs) built into commercial routers, which do not offer us the trust and flexibility we were looking for. But Linux is definitely making some headway, and has been for a while. When we covered switchdev in February 2016, it was just getting started, but now vendors like Mellanox, Broadcom, Cumulus, and Intel are writing and shipping code using the framework. Cumulus, in particular, is developing a Debian-based distribution (Cumulus Linux) that it deploys for clients on targeted hardware. Most of the hardware in that list, however, is not actually open in the more radical sense: they are devices that can run free software but are not generally open-source hardware. There are some exceptions, but they sit at the higher end of the spectrum: most organizations probably don't need a 100 Gbps ports, let alone the 128 ports in the Backpack switch that Cumulus is shipping. How much this translates to actual freedom for the end-user is therefore questionable. While I have seen encouraging progress on the high end of the hardware spectrum at Netdev, I'm not sure this will translate into immediate wins in the data center or even for home users in the short term. In the long term, however, we will hopefully see some progress in Linux's rise in general-purpose networking hardware following its dominance in general-purpose computing.

The author would like to thank the Netdev organizers for travel assistance. Also, thanks to Andrew Lunn for a technical review of this article. Note: this article first appeared in the Linux Weekly News.

DebConf17 will take place in Montreal, Canada in August 2017. We are working hard to provide fuel for hearts and minds, to make this conference once again a fertile soil for the Debian Project flourishing. Please join us and support this landmark in the Free Software calendar. Eighteen companies have already committed to sponsor DebConf17! With a warm welcome, we'd like to introduce them to you. Our first Platinum sponsor is Savoir-faire Linux, a Montreal-based Free/Open-Source Software company which offers Linux and Free Software integration solutions and actively contributes to many free software projects. "We believe that it's an essential piece [Debian], in a social and political way, to the freedom of users using modern technological systems", said Cyrille B raud, president of Savoir-faire Linux. Our first Gold sponsor is Valve, a company developing games, social entertainment platform, and game engine technologies. And our second Gold sponsor is Collabora, which offers a comprehensive range of services to help its clients to navigate the ever-evolving world of Open Source. As Silver sponsors we have credativ (a service-oriented company focusing on open-source software and also a Debian development partner), Mojatatu Networks (a Canadian company developing Software Defined Networking (SDN) solutions), the Bern University of Applied Sciences (with over 6,600 students enrolled, located in the Swiss capital), Microsoft (an American multinational technology company), Evolix (an IT managed services and support company located in Montreal), Ubuntu (the OS supported by Canonical) and Roche (a major international pharmaceutical provider and research company dedicated to personalized healthcare). ISG.EE, IBM, Bluemosh, Univention and Skroutz are our Bronze sponsors so far. And finally, The Linux foundation, R seau Koumbit and adte.ca are our supporter sponsors. Become a sponsor too! Would you like to become a sponsor? Do you know of or work in a company or organization that may consider sponsorship? Please have a look at our sponsorship brochure (or a summarized flyer), in which we outline all the details and describe the sponsor benefits. For further details, feel free to contact us through sponsors@debconf.org, and visit the DebConf17 website at https://debconf17.debconf.org.

In previous articles, we have looked at how to generate passwords and did a review of various password managers. There is, however, a third way of managing passwords other than remembering them or encrypting them in a "vault", which is what I call "password hashing". A password hasher generates site-specific passwords from a single master password using a cryptographic hash function. It thus allows a user to have a unique and secure password for every site they use while requiring no storage; they need only to remember a single password. You may know these as "deterministic or stateless password managers" but I find the "password manager" phrase to be confusing because a hasher doesn't actually store any passwords. I do not think password hashers represent a good security tradeoff so I generally do not recommend their use, unless you really do not have access to reliable storage that you can access readily. In this article, I use the word "password" for a random string used to unlock things, but "token" to represent a generated random string that the user doesn't need to remember. The input to a password hasher is a password with some site-specific context and the output from a password hasher is a token.

What is a password hasher? A password hasher uses the master password and a label (generally the host name) to generate the site-specific password. To change the generated password, the user can modify the label, for example by appending a number. Some password hashers also have different settings to generate tokens of different lengths or compositions (symbols or not, etc.) to accommodate different site-specific password policies. The whole concept of password hashers relies on the concept of one-way cryptographic hash functions or key derivation functions that take an arbitrary input string (say a password) and generate a unique token, from which it is impossible to guess the original input string. Password hashers are generally written as JavaScript bookmarklets or browser plugins and have been around for over a decade. The biggest advantage of password hashers is that you only need to remember a single password. You do not need to carry around a password manager vault: there's no "state" (other than site-specific settings, which can be easily guessed). A password hasher named Master Password makes a compelling case against traditional password managers in its documentation:

It's as though the implicit assumptions are that everybody backs all of their stuff up to at least two different devices and backups in the cloud in at least two separate countries. Well, people don't always have perfect backups. In fact, they usually don't have any.

It goes on to argue that, when you lose your password: "You lose everything. You lose your own identity." The stateless nature of password hashers also means you do not need to use cloud services to synchronize your passwords, as there is (generally, more on that later) no state to carry around. This means, for example, that the list of accounts that you have access to is only stored in your head, and not in some online database that could be hacked without your knowledge. The downside of this is, of course, that attackers do not actually need to have access to your password hasher to start cracking it: they can try to guess your master key without ever stealing anything from you other than a single token you used to log into some random web site. Password hashers also necessarily generate unique passwords for every site you use them on. While you can also do this with password managers, it is not an enforced decision. With hashers, you get distinct and strong passwords for every site with no effort.

The problem with password hashers If hashers are so great, why would you use a password manager? Programs like LessPass and Master Password seem to have strong crypto that is well implemented, so why isn't everyone using those tools? Password hashing, as a general concept, actually has serious problems: since the hashing outputs are constantly compromised (they are sent in password forms to various possibly hostile sites), it's theoretically possible to derive the master password and then break all the generated tokens in one shot. The use of stronger key derivation functions (like PBKDF2, scrypt, or HMAC) or seeds (like a profile-specific secret) makes those attacks much harder, especially if the seed is long enough to make brute-force attacks infeasible. (Unfortunately, in the case of Password Hasher Plus, the seed is derived from Math.random() calls, which are not considered cryptographically secure.) Basically, as stated by Julian Morrison in this discussion:

A password is now ciphertext, not a block of line noise. Every time you transmit it, you are giving away potential clues of use to an attacker. [...] You only have one password for all the sites, really, underneath, and it's your secret key. If it's broken, it's now a skeleton-key [...]

Newer implementations like LessPass and Master Password fix this by using reasonable key derivation algorithms (PBKDF2 and scrypt, respectively) that are more resistant to offline cracking attacks, but who knows how long those will hold? To give a concrete example, if you would like to use the new winner of the password hashing competition (Argon2) in your password manager, you can patch the program (or wait for an update) and re-encrypt your database. With a password hasher, it's not so easy: changing the algorithm means logging in to every site you visited and changing the password. As someone who used a password hasher for a few years, I can tell you this is really impractical: you quickly end up with hundreds of passwords. The LessPass developers tried to facilitate this, but they ended up mostly giving up. Which brings us to the question of state. A lot of those tools claim to work "without a server" or as being "stateless" and while those claims are partly true, hashers are way more usable (and more secure, with profile secrets) when they do keep some sort of state. For example, Password Hasher Plus records, in your browser profile, which site you visited and which settings were used on each site, which makes it easier to comply with weird password policies. But then that state needs to be backed up and synchronized across multiple devices, which led LessPass to offer a service (which you can also self-host) to keep those settings online. At this point, a key benefit of the password hasher approach (not keeping state) just disappears and you might as well use a password manager. Another issue with password hashers is choosing the right one from the start, because changing software generally means changing the algorithm, and therefore changing passwords everywhere. If there was a well-established program that was be recognized as a solid cryptographic solution by the community, I would feel more confident. But what I have seen is that there are a lot of different implementations each with its own warts and flaws; because changing is so painful, I can't actually use any of those alternatives. All of the password hashers I have reviewed have severe security versus usability tradeoffs. For example, LessPass has what seems to be a sound cryptographic implementation, but using it requires you to click on the icon, fill in the fields, click generate, and then copy the password into the field, which means at least four or five actions per password. The venerable Password Hasher is much easier to use, but it makes you type the master password directly in the site's password form, so hostile sites can simply use JavaScript to sniff the master password while it is typed. While there are workarounds implemented in Password Hasher Plus (the profile-specific secret), both tools are more or less abandoned now. The Password Hasher homepage, linked from the extension page, is now a 404. Password Hasher Plus hasn't seen a release in over a year and there is no space for collaborating on the software the homepage is simply the author's Google+ page with no information on the project. I couldn't actually find the source online and had to download the Chrome extension by hand to review the source code. Software abandonment is a serious issue for every project out there, but I would argue that it is especially severe for password hashers. Furthermore, I have had difficulty using password hashers in unified login environments like Wikipedia's or StackExchange's single-sign-on systems. Because they allow you to log in with the same password on multiple sites, you need to choose (and remember) what label you used when signing in. Did I sign in on stackoverflow.com? Or was it stackexchange.com? Also, as mentioned in the previous article about password managers, web-based password managers have serious security flaws. Since more than a few password hashers are implemented using bookmarklets, they bring all of those serious vulnerabilities with them, which can range from account name to master password disclosures. Finally, some of the password hashers use dubious crypto primitives that were valid and interesting a decade ago, but are really showing their age now. Stanford's pwdhash uses MD5, which is considered "cryptographically broken and unsuitable for further use". We have seen partial key recovery attacks against MD5 already and while those do not allow an attacker to recover the full master password yet (especially not with HMAC-MD5), I would not recommend anyone use MD5 in anything at this point, especially if changing that algorithm later is hard. Some hashers (like Password Hasher and Password Plus) use a a single round of SHA-1 to derive a token from a password; WPA2 (standardized in 2004) uses 4096 iterations of HMAC-SHA1. A recent US National Institute of Standards and Technology (NIST) report also recommends "at least 10,000 iterations of the hash function".

Conclusion Forced to suggest a password hasher, I would probably point to LessPass or Master Password, depending on the platform of the person asking. But, for now, I have determined that the security drawbacks of password hashers are not acceptable and I do not recommend them. It makes my password management recommendation shorter anyway: "remember a few carefully generated passwords and shove everything else in a password manager". [Many thanks to Daniel Kahn Gillmor for the thorough reviews provided for the password articles.]

Note: this article first appeared in the Linux Weekly News. Also, details of my research into password hashers are available in the password hashers history article.

As previously mentioned, at the end of last year I got involved with a project involving the use of 1-Wire. In particular a DS28E15 device, intended to be used as a royalty tracker for a licensed piece of hardware IP. I d no previous experience with 1-Wire (other than knowing it s commonly used for driving temperature sensors), so I took it as an opportunity to learn a bit more about it. The primary goal was to program a suitable shared key into the DS28E15 device that would also be present in the corresponding hardware device. A Maxim programmer had been ordered, but wasn t available in stock so had to be back ordered. Of course I turned to my trusty Bus Pirate, which claimed 1-Wire support. However it failed to recognise the presence of the device at all. After much head scratching I finally listened to a co-worker who had suggested it was a clock speed issue - the absence of any option to select the 1-Wire speed in the Bus Pirate or any mention of different speeds in the documentation I had read had made me doubt it was an issue. Turns out that the Bus Pirate was talking standard 1-Wire and the DS28E15 only talks overdrive 1-Wire, to the extent that it won t even announce its presence if the reset pulse conforms to the standard, rather than overdrive, reset time period. Lesson learned: listen to your co-workers sooner. A brief period of yak shaving led to adding support to the Bus Pirate for the overdrive mode (since landed in upstream), and resulted in a search request via the BP interface correctly finding the device and displaying its ROM ID. This allowed exploration of the various commands the authenticator supports, to verify that the programming sequence operated as expected. These allow for setting the shared secret, performing a SHA256 MAC against this secret and a suitable nonce, and retrieving the result. Next problem: the retrieved SHA256 MAC did not match the locally computed value. Initially endianness issues were suspected, but trying the relevant permutations did not help. Some searching found an implementation of SHA256 for the DS28E15 that showed differences between a standard SHA256 computation and what the authenticator performs. In particular SHA256 normally adds the current working state (a-g) to the current hash value (h0-h7) at the end of every block. The authenticator does this for all but the final block, where instead the hash value is set to the working state. I haven t been able to find any documentation from Maxim that this is how things are calculated, nor have I seen any generic implementation of SHA256 which supports this mode. However rolling my own C implementation based on the code I found and using it to compare the results retrieved from the device confirms that this is what s happening. So at this point we re done, right? Wait for the proper programming hardware to turn up, write the key to the devices, profit? Well, no. There was a bit of a saga involving the programmer (actually programmers, one with at least some documentation that allowed the creation of a Python tool to allow setting the key and reading + recording the ROM ID for tracking, and one with no programming documentation that came with a fancy GUI for manually doing the programming), but more importantly it was necessary to confirm that the programmed device interacted with the hardware correctly. Initial testing with the hardware was unsuccessful. Again endianness issues were considered and permutations tried, but without success. A simple key constructed to avoid such issues was tried, and things worked fine. There was a hardware simulation of both components available, so it was decided to run that and obtain a capture of the traffic between them. As the secret key was known this would then allow the random nonce to be captured, and the corresponding (correct) hash value. Tests could then be performed in software to determine what the issue was & how to generate the same hash for verification. Two sets of analyzer software were tried, OpenBench LogicSniffer (OLS) and sigrok. As it happened both failed to correctly decode the bitstream detected as 1-Wire, but were able to show the captured data graphically, allowing for decoding by eye. A slight patch to OLS to relax the timing constraints allowed it to successfully decode the full capture and provided the appropriate data for software reproduction. The end issue? A 256 bit number (as defined in VHDL) is not the same as 32 element byte array Obvious when you know what the issue is! So? What did I learn, other than a lot about 1-Wire? Firstly, don t offhandedly discount suggestions that you don t think make sense. Secondly, having a tool (in this case the Bus Pirate) that lets you easily play with a protocol via a simple interface is invaluable in understanding it. Thirdly, don t trust manufacturers to be doing something in a normal fashion when they claim to be using a well defined technology. Fourthly, be conscious about all of the different ways bitstreams can be actually processed in memory. It s not just endianness. Finally, spending the time to actually understand what s going on up front can really help when things don t work as you d expect later on - without the yak shaving to support Overdrive on the BP I wouldn t have been able to so quickly use the simulation capture to help diagnose the issue.

Short summary:My eyes hurt when I use my home desktop computer - but only with this computer. This has been very long and frustrating for me, so if you think you can help, read the whole story just below, or skip to what I've tried and what I suspect might be the problem, and post comments below, or send me a mail (my adress should be quite obvious in this page).

Whole story:Two years ago, I bought myself a new fancy motherboard (a Asus B85M-G C2) with a new fancy Intel-based processor with built-in graphics (an Intel Core i7-4770) and memory to go with it. I installed it in the place of my old AMD-based motherboard, keeping everything else (my hoard of hard drives and such) excepted the graphics card, which was not needed anymore. I immediately noticed my eyes were aching when using the computer. I was quite surprised since I had been using the screen very heavily for almost 10 years before that, without any problems. I attributed that to Intel Graphics, so I tried putting back the old graphics card, but it did not help. The situation was very frustrating, since working on the computer for an hour or so was making my eyes hurt for several days. This problem was specific to this computer, I could keep on using my computer at work and my laptop without problems.I could use the computer using SSH from my laptop, so I could profit from the faster processor, but, hey, that wasn't how it was meant to happen. I bought another screen, also tried with one from the work, without any change. I tried using two screens at the same time (this is what I have at work), also without success, so I just kept not using the computer directly. I moved recently to a new place and tried to get that working back again, but didn't get any luck. Frustrated, I got another desktop computer and another screen, and I still have the same problem ! I also tried remounting the old motherboard with the AMD processor and the old graphics card, but that didn't bring any improvement. I just don't get it. This situation is rather frustrating for me, and it's been holding me back in my software projects for two years now (which partly explains my lack of involvement in Debian over the past few years). This post is here in the hope someone will have a idea, but also for me to keep track of what I've done and what I should.

What is puzzling me is that the computer I had before was perfectly fine, and that I have a very very similar setup at work (also with a NVIDIA graphics card) that doesn't hurt my eyes at all.

What I've tried:Here is what I tried, you need to keep in mind that when a trial fails, my eyes keep on hurting for several days, and might trigger false positives.

• putting back the old (NVIDIA) graphics card, buying a new one (NVIDIA as well);
• putting back the old motherboard (but with a new OS, but maybe my eyes were too sore for a clean test);
• using another screen (a new one from the same brand, Samsung), a Dell and a HP from my work, and a brand new Phillips;
• using two screens at the same time;
• using a completely different (new, based on Xeon processors and a NVIDIA graphics card) computer (with new mouse, keyboard, hard drives and so on);
• changing house, including changing the lighting conditions, the desk, the internet provider (no, I didn't do that just because of my computer problems !);
• changing the way I drive the screens between VGA, DVI and HDMI;
• copying the system I have in my workplace to the new computer and booting from that system (after a few adaptations, though).

As you can guess, none of those brought any improvement.

Wild hypotheses:

• Is that a software thing ? Is there something wrong for me in the versions of Debian dating from August 2014 and after ?
• Is that a BIOS problem with recent computers ?
• Is that linked to some waves (bluetooth shoudln't be on, but maybe I didn't check well enough ?)
• Is that linked to EFI (but I also have the problems when I use legacy BIOS for booting)
• Something weird in my home ?
• Anything else ?

Any help will be greatly appreciated, but please don't advise going to see a doctor, I don't see how this could be a medical condition specific to my home desktop computer, unless this is a very specific psychosomatic problem.

No man pages, but almost The AWS Command Line Interface, which is available in Debian, provides no man page. Instead, that tool has an integrated help system, which allows you to run commands such as aws rds help, that, for what I have seen, generates some reStructuredText, then converts it to a man page in troff format, then calls troff to convert it to text with basic formatting, and eventually passes it to a pager. Since this is close to what man does, the result looks like a degraded man page, with some features missing such as the adaptation to the terminal width. Well, this is better than nothing, and better than what many under-documented tools can offer, but for several reasons, it still sucks: most importantly, it does not respect administrators' habits and it does not integrate with the system man database. You it does not allow you to use commands such as apropos, and you will get no man page name auto-completion from your shell since there is no man page.Generate the man pages Now, since the integrated help system does generate a man page internally, we can hack it to output it, and save it to a file:

Description: Enable a mode to generate troff man pages
 The awscli help system internally uses man pages, but only to convert
 them to text and show them with the pager. This patch enables a mode
 that prints the troff code so the user can save the man page.
 .
 To use that mode, run the help commands with an environment variable
 OUTPUT set to 'troff', for instance:
     OUTPUT='troff' aws rds help
Forwarded: no
Author: Tanguy Ortolo <tanguy+debian@ortolo.eu>
Last-Update: 2016-11-22
Index: /usr/lib/python3/dist-packages/awscli/help.py
===================================================================
--- /usr/lib/python3/dist-packages/awscli/help.py       2016-11-21 12:14:22.236254730 +0100
+++ /usr/lib/python3/dist-packages/awscli/help.py       2016-11-21 12:14:22.236254730 +0100
@@ -49,6 +49,8 @@
     Return the appropriate HelpRenderer implementation for the
     current platform.
     """
+    if 'OUTPUT' in os.environ and os.environ['OUTPUT'] == 'troff':
+        return TroffHelpRenderer()
     if platform.system() == 'Windows':
         return WindowsHelpRenderer()
     else:
@@ -97,6 +99,15 @@
         return contents
+class TroffHelpRenderer(object):
+    """
+    Render help content as troff code.
+    """
+
+    def render(self, contents):
+        sys.stdout.buffer.write(publish_string(contents, writer=manpage.Writer()))
+
+
 class PosixHelpRenderer(PagingHelpRenderer):
     """
     Render help content on a Posix-like system.  This includes

This patch must be applied from the root directory with patch -p0, otherwise GNU patch will not accept to work on files with absolute names. With that patch, you can run help commands with an environment variable OUTPUT='troff' to get the man page to use it as you like, for instance:

% OUTPUT='troff' aws rds help > aws_rds.1
% man -lt aws_rds.1   lp

Generate all the man pages Now that we are able to generate the man page of any aws command, all we need to generate all of them is a list of all the available commands. This is not that easy, because the commands are somehow derived from functions provided by a Python library named botocore, which are derived from a bunch of configuration files, and some of them are added, removed or renamed. Anyway, I have been able to write a Python script that does that, but it includes a static list of these modifications:

#! /usr/bin/python3
import subprocess
import awscli.clidriver
def write_manpage(command):
    manpage = open('%s.1' % '_'.join(command), 'w')
    command.append('help')
    process = subprocess.Popen(command,
            env= 'OUTPUT': 'troff' ,
            stdout=manpage)
    process.wait()
    manpage.close()
driver = awscli.clidriver.CLIDriver()
command_table = driver._get_command_table()
renamed_commands = \
     
        'config': 'configservice',
        'codedeploy': 'deploy',
        's3': 's3api'
     
added_commands = \
     
        's3': ['cp', 'ls', 'mb', 'mv', 'presign', 'rb', 'rm', 'sync',
               'website']
     
removed_subcommands = \
     
        'ses': ['delete-verified-email-address',
                'list-verified-email-addresses',
                'verify-email-address'],
        'ec2': ['import-instance', 'import-volume'],
        'emr': ['run-job-flow', 'describe-job-flows',
                'add-job-flow-steps', 'terminate-job-flows',
                'list-bootstrap-actions', 'list-instance-groups',
                'set-termination-protection',
                'set-visible-to-all-users'],
        'rds': ['modify-option-group']
     
added_subcommands = \
     
        'rds': ['add-option-to-option-group',
                'remove-option-from-option-group']
     
# Build a dictionary of real commands, including renames, additions and
# removals.
real_commands =  
for command in command_table:
    subcommands = []
    subcommand_table = command_table[command]._get_command_table()
    for subcommand in subcommand_table:
        # Skip removed subcommands
        if command in removed_subcommands \
                and subcommand in removed_subcommands[command]:
            continue
        subcommands.append(subcommand)
    # Add added subcommands
    if command in added_subcommands:
        for subcommand in added_subcommands[command]:
            subcommands.append(subcommand)
    # Directly add non-renamed commands
    if command not in renamed_commands:
        real_commands[command] = subcommands
    # Add renamed commands
    else:
        real_commands[renamed_commands[command]] = subcommands
# Add added commands
for command in added_commands:
    real_commands[command] = added_commands[command]
# For each real command and subcommand, generate a manpage
write_manpage(['aws'])
for command in real_commands:
    write_manpage(['aws', command])
    for subcommand in real_commands[command]:
        write_manpage(['aws', command, subcommand])
                         'sync', 'website'] 

This script will generate more than 2,000 man page files in the current directory; you will then be able to move them to /usr/local/share/man/man1. Since this is a lot of man pages, it may be appropriate to concatenate them by major command, for instance all the aws rds together

Review: The Philosopher Kings, by Jo Walton

Series:	Thessaly #2
Publisher:	Tor
Copyright:	June 2015
ISBN:	0-7653-3267-1
Format:	Hardcover
Pages:	345

Despite the cliffhanger at the end of The Just City, The Philosopher Kings doesn't pick up immediately afterwards. Argh. It's a great book (as I'm about to describe), but I really wanted to also read that book that happened in between. Still, this is the conclusion to the problem posed in The Just City, and I wouldn't recommend reading this book on its own (or, really, either book separate from the other). Despite the unwanted gap, and another change at the very start of the book that I won't state explicitly since it's a spoiler but that made me quite unhappy (despite driving the rest of the plot), this is much closer to the book that I wanted to read. Walton moves away from the SF philosophical question that drove much of the second half of The Just City in favor of going back to arguments about human organization, the nature of justice, choices between different modes of life, and the nature of human relationships. Those were the best parts of The Just City, and they're elaborated here in some fascinating ways that wouldn't have been possible in the hothouse environment of the previous book. I also thought Apollo was more interesting here than in the previous book. Still somewhat infuriating, particularly near the start, but I felt like I got more of what Walton was trying for, and more of what Apollo was attempting to use this existence to understand. And, once the plot hits its stride towards the center of the book, I started really liking Apollo. I guess it took a book and a half for him to mature enough to be interesting. A new viewpoint character, Arete, gets most of the chapters in this book, rather than following the pattern of The Just City and changing viewpoint characters every chapter. Her identity is a spoiler for The Just City, so I'll leave that a mystery. She's a bit more matter-of-fact and observational than Maia, but she does that thing that I love in Walton's characters: take an unexpected, fantastic situation, analyze and experiment with it, and draw some practical and matter-of-fact conclusions about how to proceed. I think that's the best way to describe this entire series: take a bunch of honest, thoughtful, and mostly good people, put them into a fantastic situation (at first Plato's Republic, a thought experiment made real, and then some additional fantasy complexities), and watch them tackle that situation like reasonable human beings. There is some drama, of course, because humans will disagree and will occasionally do awful, or just hurtful, things to each other. But the characters try to defuse the drama, try to be thoughtful and fair and just in their approach, and encourage change, improvement, and forgiveness in others. I don't like everyone in these books, but the vast majority of them are good people (and the few who aren't stand out), and there's something satisfying in reading about them. And the philosophical debate is wonderful throughout this book (which I'm not saying entirely because the characters have a similar reaction to a newly-introduced philosophical system as I did as a reader, although that certainly helps). I'm not saying much about the plot since so much would spoil the previous book. But Walton adds some well-done complexities and complications, and while I was dubious about them at the start of the book, I definitely came around. I enjoyed watching the characters reinvent some typical human problems, but still come at them from a unique and thoughtful angle and come up with some novel solutions. And the ending took me entirely by surprise, in a very good way. It's better than the best ending I could have imagined for the book, providing some much-needed closure and quite a bit of explanation. (And, thankfully, does not end on another cliffhanger; in fact, I'm quite curious to see what the third book is going to tackle.) Recommended, including the previous book, despite the bits that irritated me. Followed by Necessity. Rating: 9 out of 10

This is morning. Time to fly. Two boys, happy to open the hangar door and get the plane ready. It s been a year since I passed the FAA exam and became a pilot. Memories like these are my favorite reminders why I did. It is such fun to see people s faces light up with the joy of flying a few thousand feet above ground, of the beauty and freedom and peace of the skies. I ve flown 14 different passengers in that time; almost every flight I ve taken has been with people, which I enjoy. I ve heard wow or beautiful so many times, and said it myself even more times. I ve landed in two state parks, visited any number of wonderful small towns, seen historic sites and placid lakes, ascended magically over forests and plains. I ve landed at 31 airports in 10 states, flying over 13,000 miles. Not once have I encountered anyone other than friendly, kind, and outgoing. And why not? After all, we re working around magic flying carpet machines, right? (That s my brother before a flight with me, by the way) Some weeks it is easy to be glum. This week has been that way for many, myself included. But then, whether you are in the air or on the ground, if you pay attention, you realize we still live in a beautiful world with many wonderful people. And, in fact, I got a reminder of that this week. Not long after the election, I got in a plane, pushed in the throttle, and started the takeoff roll down a runway in the midst of an Indiana forest. The skies were the best kind of clear blue, and pretty soon I lifted off and could see for miles. Off in the distance, I could see the last cottony remnants of the morning s fog, lying still in the valleys, surrounding the little farms and houses as if to give them a loving hug. Wow. Sometimes the flight is bumpy. Sometimes the weather doesn t cooperate, and it doesn t happen at all. Sometimes you can fly across four large states and it feels as smooth as glass the whole way. Whatever happens, at the end of the day, the magic flying carpet machine gets locked up again. We go home, rest our heads on our soft pillows, and if we so choose, remember the beauty we experienced that day. Really, this post is not about being a pilot. This post is a reminder to pay attention to all that is beautiful in this world. It surrounds us; the smell of pine trees in the forest, the delight in the faces of children, the gentle breeze in our hair, the kind word from a stranger, the very sunrise. I hope that more of us will pay attention to the moments of clear skies and wind at our back. Even at those moments when we pull the hangar door shut.

Review: The Just City, by Jo Walton

Series:	Thessaly #1
Publisher:	Tor
Copyright:	2014
Printing:	January 2015
ISBN:	0-7653-3266-3
Format:	Hardcover
Pages:	368

The premise for The Just City is easy to state: The time-traveling goddess Athene (Athena) decides to organize and aid an attempt to create the society described in Plato's Republic. She chooses Thera (modern Santorini) before the eruption, as a safe place where this experiment wouldn't alter history. The elders of the city are seeded by people throughout history who at some point prayed to Athene, wanting to live in the Republic. The children of the age of ten that Plato suggested starting with are purchased as slaves from various points in history and transported by Athene to the island. Apollo, shaken and confused by Daphne wanting to turn into a tree rather than sleep with him, finds out about this experiment as Athene tries to explain the concept of consent to him. He decides that becoming human for a while might help him learn about volition and equal significance, and that this is the perfect location. He's one of the three viewpoint characters. The others are two women: one (Maia) from Victorian times who prayed to Athene in a moment of longing for the tentative sexual equality of the Republic and was recruited as one of the elders, and another (Simmea) who is bought as a slave and becomes one of the children. I should admit up-front that I've never read Plato's Republic, or indeed much of Plato at all, just small bits for classes. The elders (and of course the gods) all have, and are attempting to stick quite closely to Plato's outline of the ideal city. The children haven't, though, so the book is quite readable for people like me who only remember a few vague aspects of Plato's vision from school. The reader learns the principles alongside Simmea. One of Walton's strengths is taking a science fiction concept, putting real people into it, and letting the quotidian mingle with the fantastic. Simmea is my favorite character here: her journey to the city is deeply traumatic, but the opportunity she gets there is incredible and unforeseen, and she comes to love the city while still understanding, and arguing about, its possible flaws. Maia is nearly as successful; Walton does a good job with committee debates and discussions, avoids coming down too heavily on the drama, and shows a believable picture of people with very different backgrounds and beliefs coming together to flesh out the outlines of something they all agree with, or at least want to try. I found Apollo less engaging as a character, partly because I never quite understood his motives or his weird failure to understand the principles of consent. Walton doesn't portray him as either hopelessly arrogant or hopelessly narcissistic, which would have been easy outs, but in avoiding those two obvious explanations for his failures of empathy, I felt like she left him with an odd and unexplained hole in his personality. He's a weirdly passive half-character for much of the book, although he does develop a bit more towards the end (which was probably the point). Half the fun of this book is working out what the Republic would be like in practice, and what breakdowns and compromises would happen as soon as you put real people in it. Athene obviously has to do a bit of cheating to make a utopia invented as an intellectual exercise work out in practice, plus a bit more for comfort (electricity and indoor plumbing, for instance). The most substantial cheat is robots to replace slaves and do quite a bit that slaves couldn't. Birth control (something Plato obviously never would have thought of) is another notable cheat; it's postulated to be an ancient method since lost, but even if that existed, there's no way it would be this reliable. But otherwise, the society mostly works, and Walton shows enough of the arguing and mechanics to make that believable, while still avoiding infodumps and boring descriptions. It's neatly done, although I'm still a bit dubious that the elders from later eras would have put up with the primitive conditions with this little complaint. The novel needs a plot, of course, and that's the other half of the fun. I can't talk about this in any detail without spoiling the book, since the plot only kicks in about halfway through once the setup and character introductions are complete. That makes it hard to explain why I found this a bit less successful, although parts of it are brilliant. What worked for me is the growth of Simmea and her friends as students and philosophers, the arguments and discussions (and their growing enthusiasm for argument and discussion), and the way Greek mythology is woven subtly and undramatically into the story. It really does feel like sitting in on ancient Greek philosophical arguments and experiments, and by that measure Walton has succeeded admirably in her goal. What didn't work for me was the driving conflict of the story, once it's introduced. I can't describe it without spoilers, but it's an old trope in science fiction and one with little scientific basis. It may seem weird to argue that point in a book with time-traveling Greek gods, a literal Lethe, and a Greek idea of souls, but those are mythological background material. The SF trope is something about which I have personal expertise and which simply doesn't work that way, and I had a harder time getting past that than alternate metaphysical properties. It threw me out of the book a bit. I see why Walton chose the conflict she did, but I felt like she could have gotten to the same place in the plot, admittedly with more difficulty, by using some of the more dubious aspects of Plato's long-term plan plus some other obstacles that were already built into the world. This more direct approach added a bit of SF-style analysis of the unknown that seemed weirdly at odds with the rest of the story (even if the delight of one of the characters is endearing). That complaint aside, I really enjoyed reading this book. Apollo didn't entirely work for me, but all of the other characters are excellent, and Walton keeps the story moving at a comfortable clip. Given the amount of description required, particularly for an audience that may not have read the Republic, a lesser writer could have easily slipped into the infodump trap. Walton never does. Fair warning, though: The Just City does end on a cliffhanger, and is in no way a standalone novel. You will probably want to have the sequel on hand. Followed by The Philosopher Kings. Rating: 7 out of 10

In the last week I played a bit with UDD (Ultimate Debian Database). After some experiments I did a script to generate a daily report about source packages in Debian. This report is useful to choose a package that needs help. The daily report has six sections:

• Sources in Debian Sid (including orphan)
• Sources in Debian Sid (only Orphan, RFA or RFH)
• Top 200 sources in Debian Sid with outdated Standards-Version
• Top 200 sources in Debian Sid with NMUs
• Top 200 sources in Debian Sid with BUGs
• Top 200 sources in Debian Sid with RC BUGs

The first section has several important data about all source packages in Debian, ordered by last upload to Sid. It is very useful to see packages without revisions for a long time. Other interesting data about each package are Standards-Version, packaging format, number of NMUs, among others. Believe it or not, there are packages uploaded to Sid for the last time 2003! (seven packages) With the report, you can choose a ideal package to do QA uploads, NMUs or to adopt. Well, if you like to review packages, this report is for you: https://people.debian.org/~eriberto/eriberto_stats.html. Enjoy!