Search Results: "mjr"

17 May 2020

Matthew Palmer: Private Key Redaction: UR DOIN IT RONG

Because posting private keys on the Internet is a bad idea, some people like to redact their private keys, so that it looks kinda-sorta like a private key, but it isn t actually giving away anything secret. Unfortunately, due to the way that private keys are represented, it is easy to redact a key in such a way that it doesn t actually redact anything at all. RSA private keys are particularly bad at this, but the problem can (potentially) apply to other keys as well. I ll show you a bit of Inside Baseball with key formats, and then demonstrate the practical implications. Finally, we ll go through a practical worked example from an actual not-really-redacted key I recently stumbled across in my travels.

The Private Lives of Private Keys Here is what a typical private key looks like, when you come across it:
-----BEGIN RSA PRIVATE KEY-----
MGICAQACEQCxjdTmecltJEz2PLMpS4BXAgMBAAECEDKtuwD17gpagnASq1zQTYEC
CQDVTYVsjjF7IQIJANUYZsIjRsR3AgkAkahDUXL0RSECCB78r2SnsJC9AghaOK3F
sKoELg==
-----END RSA PRIVATE KEY-----
Obviously, there s some hidden meaning in there computers don t encrypt things by shouting BEGIN RSA PRIVATE KEY! , after all. What is between the BEGIN/END lines above is, in fact, a base64-encoded DER format ASN.1 structure representing a PKCS#1 private key. In simple terms, it s a list of numbers very important numbers. The list of numbers is, in order:
  • A version number (0);
  • The public modulus , commonly referred to as n ;
  • The public exponent , or e (which is almost always 65,537, for various unimportant reasons);
  • The private exponent , or d ;
  • The two private primes , or p and q ;
  • Two exponents, which are known as dmp1 and dmq1 ; and
  • A coefficient, known as iqmp .

Why Is This a Problem? The thing is, only three of those numbers are actually required in a private key. The rest, whilst useful to allow the RSA encryption and decryption to be more efficient, aren t necessary. The three absolutely required values are e, p, and q. Of the other numbers, most of them are at least about the same size as each of p and q. So of the total data in an RSA key, less than a quarter of the data is required. Let me show you with the above toy key, by breaking it down piece by piece1:
  • MGI DER for this is a sequence
  • CAQ version (0)
  • CxjdTmecltJEz2PLMpS4BX n
  • AgMBAA e
  • ECEDKtuwD17gpagnASq1zQTY d
  • ECCQDVTYVsjjF7IQ p
  • IJANUYZsIjRsR3 q
  • AgkAkahDUXL0RS dmp1
  • ECCB78r2SnsJC9 dmq1
  • AghaOK3FsKoELg== iqmp
Remember that in order to reconstruct all of these values, all I need are e, p, and q and e is pretty much always 65,537. So I could redact almost all of this key, and still give all the important, private bits of this key. Let me show you:
-----BEGIN RSA PRIVATE KEY-----
..............................................................EC
CQDVTYVsjjF7IQIJANUYZsIjRsR3....................................
........
-----END RSA PRIVATE KEY-----
Now, I doubt that anyone is going to redact a key precisely like this but then again, this isn t a typical RSA key. They usually look a lot more like this:
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAu6Inch7+mWtKn+leB9uCG3MaJIxRyvC/5KTz2fR+h+GOhqj4
SZJobiVB4FrE5FgC7AnlH6qeRi9MI0s6dt5UWZ5oNIeWSaOOeNO+EJDUkSVf67wj
SNGXlSjGAkPZ0nRJiDjhuPvQmdW53hOaBLk5udxPEQbenpXAzbLJ7wH5ouLQ3nQw
HwpwDNQhF6zRO8WoscpDVThOAM+s4PS7EiK8ZR4hu2toon8Ynadlm95V45wR0VlW
zywgbkZCKa1IMrDCscB6CglQ10M3Xzya3iTzDtQxYMVqhDrA7uBYRxA0y1sER+Rb
yhEh03xz3AWemJVLCQuU06r+FABXJuY/QuAVvQIDAQABAoIBAFqwWVhzWqNUlFEO
PoCVvCEAVRZtK+tmyZj9kU87ORz8DCNR8A+/T/JM17ZUqO2lDGSBs9jGYpGRsr8s
USm69BIM2ljpX95fyzDjRu5C0jsFUYNi/7rmctmJR4s4uENcKV5J/++k5oI0Jw4L
c1ntHNWUgjK8m0UTJIlHbQq0bbAoFEcfdZxd3W+SzRG3jND3gifqKxBG04YDwloy
tu+bPV2jEih6p8tykew5OJwtJ3XsSZnqJMwcvDciVbwYNiJ6pUvGq6Z9kumOavm9
XU26m4cWipuK0URWbHWQA7SjbktqEpxsFrn5bYhJ9qXgLUh/I1+WhB2GEf3hQF5A
pDTN4oECgYEA7Kp6lE7ugFBDC09sKAhoQWrVSiFpZG4Z1gsL9z5YmZU/vZf0Su0n
9J2/k5B1GghvSwkTqpDZLXgNz8eIX0WCsS1xpzOuORSNvS1DWuzyATIG2cExuRiB
jYWIJUeCpa5p2PdlZmBrnD/hJ4oNk4oAVpf+HisfDSN7HBpN+TJfcAUCgYEAyvY7
Y4hQfHIdcfF3A9eeCGazIYbwVyfoGu70S/BZb2NoNEPymqsz7NOfwZQkL4O7R3Wl
Rm0vrWT8T5ykEUgT+2ruZVXYSQCKUOl18acbAy0eZ81wGBljZc9VWBrP1rHviVWd
OVDRZNjz6nd6ZMrJvxRa24TvxZbJMmO1cgSW1FkCgYAoWBd1WM9HiGclcnCZknVT
UYbykCeLO0mkN1Xe2/32kH7BLzox26PIC2wxF5seyPlP7Ugw92hOW/zewsD4nLze
v0R0oFa+3EYdTa4BvgqzMXgBfvGfABJ1saG32SzoWYcpuWLLxPwTMsCLIPmXgRr1
qAtl0SwF7Vp7O/C23mNukQKBgB89DOEB7xloWv3Zo27U9f7nB7UmVsGjY8cZdkJl
6O4LB9PbjXCe3ywZWmJqEbO6e83A3sJbNdZjT65VNq9uP50X1T+FmfeKfL99X2jl
RnQTsrVZWmJrLfBSnBkmb0zlMDAcHEnhFYmHFuvEnfL7f1fIoz9cU6c+0RLPY/L7
n9dpAoGAXih17mcmtnV+Ce+lBWzGWw9P4kVDSIxzGxd8gprrGKLa3Q9VuOrLdt58
++UzNUaBN6VYAe4jgxGfZfh+IaSlMouwOjDgE/qzgY8QsjBubzmABR/KWCYiRqkj
qpWCgo1FC1Gn94gh/+dW2Q8+NjYtXWNqQcjRP4AKTBnPktEvdMA=
-----END RSA PRIVATE KEY-----
People typically redact keys by deleting whole lines, and usually replacing them with [...] and the like. But only about 345 of those 1588 characters (excluding the header and footer) are required to construct the entire key. You can redact about 4/5ths of that giant blob of stuff, and your private parts (or at least, those of your key) are still left uncomfortably exposed.

But Wait! There s More! Remember how I said that everything in the key other than e, p, and q could be derived from those three numbers? Let s talk about one of those numbers: n. This is known as the public modulus (because, along with e, it is also present in the public key). It is very easy to calculate: n = p * q. It is also very early in the key (the second number, in fact). Since n = p * q, it follows that q = n / p. Thus, as long as the key is intact up to p, you can derive q by simple division.

Real World Redaction At this point, I d like to introduce an acquaintance of mine: Mr. Johan Finn. He is the proud owner of the GitHub repo johanfinn/scripts. For a while, his repo contained a script that contained a poorly-redacted private key. He since deleted it, by making a new commit, but of course because git never really deletes anything, it s still available. Of course, Mr. Finn may delete the repo, or force-push a new history without that commit, so here is the redacted private key, with a bit of the surrounding shell script, for our illustrative pleasure:
#Add private key to .ssh folder
cd /home/johan/.ssh/
echo  "-----BEGIN RSA PRIVATE KEY-----
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
 
MIIJKgIBAAKCAgEAxEVih1JGb8gu/Fm4AZh+ZwJw/pjzzliWrg4mICFt1g7SmIE2
TCQMKABdwd11wOFKCPc/UzRH/fHuQcvWrpbOSdqev/zKff9iedKw/YygkMeIRaXB
fYELqvUAOJ8PPfDm70st9GJRhjGgo5+L3cJB2gfgeiDNHzaFvapRSU0oMGQX+kI9
ezsjDAn+0Pp+r3h/u1QpLSH4moRFGF4omNydI+3iTGB98/EzuNhRBHRNq4oBV5SG
Pq/A1bem2ninnoEaQ+OPESxYzDz3Jy9jV0W/6LvtJ844m+XX69H5fqq5dy55z6DW
sGKn78ULPVZPsYH5Y7C+CM6GAn4nYCpau0t52sqsY5epXdeYx4Dc+Wm0CjXrUDEe
Egl4loPKDxJkQqQ/MQiz6Le/UK9vEmnWn1TRXK3ekzNV4NgDfJANBQobOpwt8WVB
rbsC0ON7n680RQnl7PltK9P1AQW5vHsahkoixk/BhcwhkrkZGyDIl9g8Q/Euyoq3
eivKPLz7/rhDE7C1BzFy7v8AjC3w7i9QeHcWOZFAXo5hiDasIAkljDOsdfD4tP5/
wSO6E6pjL3kJ+RH2FCHd7ciQb+IcuXbku64ln8gab4p8jLa/mcMI+V3eWYnZ82Yu
axsa85hAe4wb60cp/rCJo7ihhDTTvGooqtTisOv2nSvCYpcW9qbL6cGjAXECAwEA
AQKCAgEAjz6wnWDP5Y9ts2FrqUZ5ooamnzpUXlpLhrbu3m5ncl4ZF5LfH+QDN0Kl
KvONmHsUhJynC/vROybSJBU4Fu4bms1DJY3C39h/L7g00qhLG7901pgWMpn3QQtU
4P49qpBii20MGhuTsmQQALtV4kB/vTgYfinoawpo67cdYmk8lqzGzzB/HKxZdNTq
s+zOfxRr7PWMo9LyVRuKLjGyYXZJ/coFaobWBi8Y96Rw5NZZRYQQXLIalC/Dhndm
AHckpstEtx2i8f6yxEUOgPvV/gD7Akn92RpqOGW0g/kYpXjGqZQy9PVHGy61sInY
HSkcOspIkJiS6WyJY9JcvJPM6ns4b84GE9qoUlWVF3RWJk1dqYCw5hz4U8LFyxsF
R6WhYiImvjxBLpab55rSqbGkzjI2z+ucDZyl1gqIv9U6qceVsgRyuqdfVN4deU22
LzO5IEDhnGdFqg9KQY7u8zm686Ejs64T1sh0y4GOmGsSg+P6nsqkdlXH8C+Cf03F
lqPFg8WQC7ojl/S8dPmkT5tcJh3BPwIWuvbtVjFOGQc8x0lb+NwK8h2Nsn6LNazS
0H90adh/IyYX4sBMokrpxAi+gMAWiyJHIHLeH2itNKtAQd3qQowbrWNswJSgJzsT
JuJ7uqRKAFkE6nCeAkuj/6KHHMPsfCAffVdyGaWqhoxmPOrnVgECggEBAOrCCwiC
XxwUgjOfOKx68siFJLfHf4vPo42LZOkAQq5aUmcWHbJVXmoxLYSczyAROopY0wd6
Dx8rqnpO7OtZsdJMeBSHbMVKoBZ77hiCQlrljcj12moFaEAButLCdZFsZW4zF/sx
kWIAaPH9vc4MvHHyvyNoB3yQRdevu57X7xGf9UxWuPil/jvdbt9toaraUT6rUBWU
GYPNKaLFsQzKsFWAzp5RGpASkhuiBJ0Qx3cfLyirjrKqTipe3o3gh/5RSHQ6VAhz
gdUG7WszNWk8FDCL6RTWzPOrbUyJo/wz1kblsL3vhV7ldEKFHeEjsDGroW2VUFlS
asAHNvM4/uYcOSECggEBANYH0427qZtLVuL97htXW9kCAT75xbMwgRskAH4nJDlZ
IggDErmzBhtrHgR+9X09iL47jr7dUcrVNPHzK/WXALFSKzXhkG/yAgmt3r14WgJ6
5y7010LlPFrzaNEyO/S4ISuBLt4cinjJsrFpoo0WI8jXeM5ddG6ncxdurKXMymY7
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::.::
:::::::::::::::::::::::::::.::::::::::::::::::::::::::::::::::::
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLlL
 
 
 
YYYYYYYYYYYYYYYYYYYYYyYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
gff0GJCOMZ65pMSy3A3cSAtjlKnb4fWzuHD5CFbusN4WhCT/tNxGNSpzvxd8GIDs
nY7exs9L230oCCpedVgcbayHCbkChEfoPzL1e1jXjgCwCTgt8GjeEFqc1gXNEaUn
O8AJ4VlR8fRszHm6yR0ZUBdY7UJddxQiYOzt0S1RLlECggEAbdcs4mZdqf3OjejJ
06oTPs9NRtAJVZlppSi7pmmAyaNpOuKWMoLPElDAQ3Q7VX26LlExLCZoPOVpdqDH
KbdmBEfTR4e11Pn9vYdu9/i6o10U4hpmf4TYKlqk10g1Sj21l8JATj/7Diey8scO
sAI1iftSg3aBSj8W7rxCxSezrENzuqw5D95a/he1cMUTB6XuravqZK5O4eR0vrxR
AvMzXk5OXrUEALUvt84u6m6XZZ0pq5XZxq74s8p/x1JvTwcpJ3jDKNEixlHfdHEZ
ZIu/xpcwD5gRfVGQamdcWvzGHZYLBFO1y5kAtL8kI9tW7WaouWVLmv99AyxdAaCB
Y5mBAQKCAQEAzU7AnorPzYndlOzkxRFtp6MGsvRBsvvqPLCyUFEXrHNV872O7tdO
GmsMZl+q+TJXw7O54FjJJvqSSS1sk68AGRirHop7VQce8U36BmI2ZX6j2SVAgIkI
9m3btCCt5rfiCatn2+Qg6HECmrCsHw6H0RbwaXS4RZUXD/k4X+sslBitOb7K+Y+N
Bacq6QxxjlIqQdKKPs4P2PNHEAey+kEJJGEQ7bTkNxCZ21kgi1Sc5L8U/IGy0BMC
PvJxssLdaWILyp3Ws8Q4RAoC5c0ZP0W2j+5NSbi3jsDFi0Y6/2GRdY1HAZX4twem
Q0NCedq1JNatP1gsb6bcnVHFDEGsj/35oQKCAQEAgmWMuSrojR/fjJzvke6Wvbox
FRnPk+6YRzuYhAP/YPxSRYyB5at++5Q1qr7QWn7NFozFIVFFT8CBU36ktWQ39MGm
cJ5SGyN9nAbbuWA6e+/u059R7QL+6f64xHRAGyLT3gOb1G0N6h7VqFT25q5Tq0rc
Lf/CvLKoudjv+sQ5GKBPT18+zxmwJ8YUWAsXUyrqoFWY/Tvo5yLxaC0W2gh3+Ppi
EDqe4RRJ3VKuKfZxHn5VLxgtBFN96Gy0+Htm5tiMKOZMYAkHiL+vrVZAX0hIEuRZ
JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
-----END RSA PRIVATE KEY-----" >> id_rsa
Now, if you try to reconstruct this key by removing the obvious garbage lines (the ones that are all repeated characters, some of which aren t even valid base64 characters), it still isn t a key at least, openssl pkey doesn t want anything to do with it. The key is very much still in there, though, as we shall soon see. Using a gem I wrote and a quick bit of Ruby, we can extract a complete private key. The irb session looks something like this:
>> require "derparse"
>> b64 = <<EOF
MIIJKgIBAAKCAgEAxEVih1JGb8gu/Fm4AZh+ZwJw/pjzzliWrg4mICFt1g7SmIE2
TCQMKABdwd11wOFKCPc/UzRH/fHuQcvWrpbOSdqev/zKff9iedKw/YygkMeIRaXB
fYELqvUAOJ8PPfDm70st9GJRhjGgo5+L3cJB2gfgeiDNHzaFvapRSU0oMGQX+kI9
ezsjDAn+0Pp+r3h/u1QpLSH4moRFGF4omNydI+3iTGB98/EzuNhRBHRNq4oBV5SG
Pq/A1bem2ninnoEaQ+OPESxYzDz3Jy9jV0W/6LvtJ844m+XX69H5fqq5dy55z6DW
sGKn78ULPVZPsYH5Y7C+CM6GAn4nYCpau0t52sqsY5epXdeYx4Dc+Wm0CjXrUDEe
Egl4loPKDxJkQqQ/MQiz6Le/UK9vEmnWn1TRXK3ekzNV4NgDfJANBQobOpwt8WVB
rbsC0ON7n680RQnl7PltK9P1AQW5vHsahkoixk/BhcwhkrkZGyDIl9g8Q/Euyoq3
eivKPLz7/rhDE7C1BzFy7v8AjC3w7i9QeHcWOZFAXo5hiDasIAkljDOsdfD4tP5/
wSO6E6pjL3kJ+RH2FCHd7ciQb+IcuXbku64ln8gab4p8jLa/mcMI+V3eWYnZ82Yu
axsa85hAe4wb60cp/rCJo7ihhDTTvGooqtTisOv2nSvCYpcW9qbL6cGjAXECAwEA
AQKCAgEAjz6wnWDP5Y9ts2FrqUZ5ooamnzpUXlpLhrbu3m5ncl4ZF5LfH+QDN0Kl
KvONmHsUhJynC/vROybSJBU4Fu4bms1DJY3C39h/L7g00qhLG7901pgWMpn3QQtU
4P49qpBii20MGhuTsmQQALtV4kB/vTgYfinoawpo67cdYmk8lqzGzzB/HKxZdNTq
s+zOfxRr7PWMo9LyVRuKLjGyYXZJ/coFaobWBi8Y96Rw5NZZRYQQXLIalC/Dhndm
AHckpstEtx2i8f6yxEUOgPvV/gD7Akn92RpqOGW0g/kYpXjGqZQy9PVHGy61sInY
HSkcOspIkJiS6WyJY9JcvJPM6ns4b84GE9qoUlWVF3RWJk1dqYCw5hz4U8LFyxsF
R6WhYiImvjxBLpab55rSqbGkzjI2z+ucDZyl1gqIv9U6qceVsgRyuqdfVN4deU22
LzO5IEDhnGdFqg9KQY7u8zm686Ejs64T1sh0y4GOmGsSg+P6nsqkdlXH8C+Cf03F
lqPFg8WQC7ojl/S8dPmkT5tcJh3BPwIWuvbtVjFOGQc8x0lb+NwK8h2Nsn6LNazS
0H90adh/IyYX4sBMokrpxAi+gMAWiyJHIHLeH2itNKtAQd3qQowbrWNswJSgJzsT
JuJ7uqRKAFkE6nCeAkuj/6KHHMPsfCAffVdyGaWqhoxmPOrnVgECggEBAOrCCwiC
XxwUgjOfOKx68siFJLfHf4vPo42LZOkAQq5aUmcWHbJVXmoxLYSczyAROopY0wd6
Dx8rqnpO7OtZsdJMeBSHbMVKoBZ77hiCQlrljcj12moFaEAButLCdZFsZW4zF/sx
kWIAaPH9vc4MvHHyvyNoB3yQRdevu57X7xGf9UxWuPil/jvdbt9toaraUT6rUBWU
GYPNKaLFsQzKsFWAzp5RGpASkhuiBJ0Qx3cfLyirjrKqTipe3o3gh/5RSHQ6VAhz
gdUG7WszNWk8FDCL6RTWzPOrbUyJo/wz1kblsL3vhV7ldEKFHeEjsDGroW2VUFlS
asAHNvM4/uYcOSECggEBANYH0427qZtLVuL97htXW9kCAT75xbMwgRskAH4nJDlZ
IggDErmzBhtrHgR+9X09iL47jr7dUcrVNPHzK/WXALFSKzXhkG/yAgmt3r14WgJ6
5y7010LlPFrzaNEyO/S4ISuBLt4cinjJsrFpoo0WI8jXeM5ddG6ncxdurKXMymY7
EOF
>> b64 += <<EOF
gff0GJCOMZ65pMSy3A3cSAtjlKnb4fWzuHD5CFbusN4WhCT/tNxGNSpzvxd8GIDs
nY7exs9L230oCCpedVgcbayHCbkChEfoPzL1e1jXjgCwCTgt8GjeEFqc1gXNEaUn
O8AJ4VlR8fRszHm6yR0ZUBdY7UJddxQiYOzt0S1RLlECggEAbdcs4mZdqf3OjejJ
06oTPs9NRtAJVZlppSi7pmmAyaNpOuKWMoLPElDAQ3Q7VX26LlExLCZoPOVpdqDH
KbdmBEfTR4e11Pn9vYdu9/i6o10U4hpmf4TYKlqk10g1Sj21l8JATj/7Diey8scO
sAI1iftSg3aBSj8W7rxCxSezrENzuqw5D95a/he1cMUTB6XuravqZK5O4eR0vrxR
AvMzXk5OXrUEALUvt84u6m6XZZ0pq5XZxq74s8p/x1JvTwcpJ3jDKNEixlHfdHEZ
ZIu/xpcwD5gRfVGQamdcWvzGHZYLBFO1y5kAtL8kI9tW7WaouWVLmv99AyxdAaCB
Y5mBAQKCAQEAzU7AnorPzYndlOzkxRFtp6MGsvRBsvvqPLCyUFEXrHNV872O7tdO
GmsMZl+q+TJXw7O54FjJJvqSSS1sk68AGRirHop7VQce8U36BmI2ZX6j2SVAgIkI
9m3btCCt5rfiCatn2+Qg6HECmrCsHw6H0RbwaXS4RZUXD/k4X+sslBitOb7K+Y+N
Bacq6QxxjlIqQdKKPs4P2PNHEAey+kEJJGEQ7bTkNxCZ21kgi1Sc5L8U/IGy0BMC
PvJxssLdaWILyp3Ws8Q4RAoC5c0ZP0W2j+5NSbi3jsDFi0Y6/2GRdY1HAZX4twem
Q0NCedq1JNatP1gsb6bcnVHFDEGsj/35oQKCAQEAgmWMuSrojR/fjJzvke6Wvbox
FRnPk+6YRzuYhAP/YPxSRYyB5at++5Q1qr7QWn7NFozFIVFFT8CBU36ktWQ39MGm
cJ5SGyN9nAbbuWA6e+/u059R7QL+6f64xHRAGyLT3gOb1G0N6h7VqFT25q5Tq0rc
Lf/CvLKoudjv+sQ5GKBPT18+zxmwJ8YUWAsXUyrqoFWY/Tvo5yLxaC0W2gh3+Ppi
EDqe4RRJ3VKuKfZxHn5VLxgtBFN96Gy0+Htm5tiMKOZMYAkHiL+vrVZAX0hIEuRZ
EOF
>> der = b64.unpack("m").first
>> c = DerParse.new(der).first_node.first_child
>> version = c.value
=> 0
>> c = c.next_node
>> n = c.value
=> 80071596234464993385068908004931... # (etc)
>> c = c.next_node
>> e = c.value
=> 65537
>> c = c.next_node
>> d = c.value
=> 58438813486895877116761996105770... # (etc)
>> c = c.next_node
>> p = c.value
=> 29635449580247160226960937109864... # (etc)
>> c = c.next_node
>> q = c.value
=> 27018856595256414771163410576410... # (etc)
What I ve done, in case you don t speak Ruby, is take the two chunks of plausible-looking base64 data, chuck them together into a variable named b64, unbase64 it into a variable named der, pass that into a new DerParse instance, and then walk the DER value tree until I got all the values I need. Interestingly, the q value actually traverses the split in the two chunks, which means that there s always the possibility that there are lines missing from the key. However, since p and q are supposed to be prime, we can sanity check them to see if corruption is likely to have occurred:
>> require "openssl"
>> OpenSSL::BN.new(p).prime?
=> true
>> OpenSSL::BN.new(q).prime?
=> true
Excellent! The chances of a corrupted file producing valid-but-incorrect prime numbers isn t huge, so we can be fairly confident that we ve got the real p and q. Now, with the help of another one of my creations we can use e, p, and q to create a fully-operational battle key:
>> require "openssl/pkey/rsa"
>> k = OpenSSL::PKey::RSA.from_factors(p, q, e)
=> #<OpenSSL::PKey::RSA:0x0000559d5903cd38>
>> k.valid?
=> true
>> k.verify(OpenSSL::Digest::SHA256.new, k.sign(OpenSSL::Digest::SHA256.new, "bob"), "bob")
=> true
and there you have it. One fairly redacted-looking private key brought back to life by maths and far too much free time. Sorry Mr. Finn, I hope you re not still using that key on anything Internet-facing.

What About Other Key Types? EC keys are very different beasts, but they have much the same problems as RSA keys. A typical EC key contains both private and public data, and the public portion is twice the size so only about 1/3 of the data in the key is private material. It is quite plausible that you can redact an EC key and leave all the actually private bits exposed.

What Do We Do About It? In short: don t ever try and redact real private keys. For documentation purposes, just put KEY GOES HERE in the appropriate spot, or something like that. Store your secrets somewhere that isn t a public (or even private!) git repo. Generating a dummy private key and sticking it in there isn t a great idea, for different reasons: people have this odd habit of reusing demo keys in real life. There s no need to encourage that sort of thing.
  1. Technically the pieces aren t 100% aligned with the underlying DER, because of how base64 works. I felt it was easier to understand if I stuck to chopping up the base64, rather than decoding into DER and then chopping up the DER.

13 April 2020

Giovanni Mascellani: DKIM for Debian Developers

What is DKIM? DKIM (DomainKeys Identified Mail), as Wikipedia puts it, "is an email authentication method designed to detect forged sender addresses in emails (email spoofing), a technique often used in phishing and email spam". More prosaically, one of the reasons email spam is so abundant is that, given a certain email message, there is no simple way to know for certain who sent it and how reputable they are. So even if people having addresses @debian.org are very nice and well-behaving, any random spammer can easily send emails from whatever@debian.org, and even if you trust people from @debian.org you cannot easily configure your antispam filter to just accept all emails from @debian.org, because spammers would get in too. Since nearly ten years DKIM is there to help you. If you send an email from @debian.org with DKIM, it will have a header like this:
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=debian.org;
    s=vps.gio.user; t=1586779391;
    bh=B6tckJy2cynGjNRdm3lhFDrp0tD7fF8hS4x0FCfLADo=;
    h=From:Subject:To:Date:From;
    b=H4EDlATxVm7XNqPy2x7IqCchBUz1SxFtUSstB23BAsdyTKJIohM0O4RRWhrQX+pqE
     prPVhzcfNALMwlfExNE69940Q6pMCuYsoxNQjU7Jl/UX1q6PGqdVSO+mKv/aEI+N49
     vvYNgPJNLaAFnYqbWCPI8mNskLHLe2VFYjSjE4GJFOxl9o2Gpe9f5035FYPJ/hnqBF
     XPnZq7Osd9UtBrBq8agEooTCZHbNFSyiXdS0qp1ts7HAo/rfrBfbQSk39fOOQ5GbjV
     6FehkN4GAXFNoFnjfmjrVDJC6hvA8m0tJHbmZrNQS0ljG/SyffW4OTlzFzu4jOmDNi
     UHLnEgT07eucw==
The field d=debian.org is the domain this email claims to be from and the fields bh= and b= are a cryptographic public key signature certifying this fact. How do I check that the email is actually from @debian.org? I use the selector s=vps.gio.user to fetch the public key via DNS, and then use the public key to verify the signature.
$ host -t TXT vps.gio.user._domainkey.debian.org
vps.gio.user._domainkey.debian.org descriptive text "v=DKIM1; k=rsa; s=email; h=sha256; p=" "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsM/W/kxtKWT58Eak0cfm/ntvurfbkkvugrG2jfvSMnHHkFyfJ34Xvn/HhQPLwX1QsjhuLV+tW+BQtxY7jxSABCee6nHQRBrpDej1t86ubw3CSrxcg1mzJI5BbL8un0cwYoBtUvhCYAZKarv1W2otCGs43L0s" "GtEqqtmYN/hIVVm4FcqeYS1cYrZxDsjPzCEocpYBhqHh1MTeUEddVmPHKZswzvllaWF0mgIXrfDNAE0LiX39aFKWtgvflrYFKiL4hCDnBcP2Mr71TVblfDY0wEdAEbGEJqHR1SxvWyn0UU1ZL4vTcylB/KJuV2gMhznOjbnQ6cjAhr2JYpweTYzz3wIDAQAB"
There it is! Debian declares in its DNS record that that key is authorized to sign outbound email from @debian.org. The spammer hopefully does not have access to Debian's DKIM keys, and they cannot sign emails. Many large and small email services have already deployed DKIM since years, while most @debian.org emails still do not use it. Why not? Because people send @debian.org emails from many different servers. Basically, every DD used their @debian.org address sends email from their own mail server, and those mail servers (fortunately) do not have access to Debian's DNS record to install their DKIM keys. Well, that was true until yesterday! :-) A few weeks ago I poked DSA asking to allow any Debian Developer to install their DKIM keys, so that DDs could use DKIM to sign their emails and hopefully reduce the amount of spam sent from @debian.org. They have done it (thank you DSA very much, especially adsb), and now it is possible to use it! How do I configure it? I will not write here a full DKIM tutorial, there are many around. You have to use opendkim-genkey to generate a key and then configure your mail server to use opendkim to digitally sign outbound email. There are a few Debian-specific things you have to care about, though. First the have to choose a selector, which is a string used to distinguish many DKIM keys belonging to the same domain. Debian allows you to installa a key whose selector is <something>.<uid>.user, where <uid> is your Debian uid (this is done both for namespacing reasons and for exposing who might be abusing the system). So check carefully that your selector has this form. Then you cannot edit directly Debian's DNS record. But you can use the email-LDAP gateway on db.debian.org to install your key in a way similar to how entries in debian.net are handled (see the updated documentation). Specifically, suppose that opendkim-genkey generated the following thing for selector vps.gio.user and domain debian.org:
vps.gio.user._domainkey IN  TXT ( "v=DKIM1; h=sha256; k=rsa; "
      "p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsM/W/kxtKWT58Eak0cfm/ntvurfbkkvugrG2jfvSMnHHkFyfJ34Xvn/HhQPLwX1QsjhuLV+tW+BQtxY7jxSABCee6nHQRBrpDej1t86ubw3CSrxcg1mzJI5BbL8un0cwYoBtUvhCYAZKarv1W2otCGs43L0sGtEqqtmYN/hIVVm4FcqeYS1cYrZxDsjPzCEocpYBhqHh1MTeUE"
      "ddVmPHKZswzvllaWF0mgIXrfDNAE0LiX39aFKWtgvflrYFKiL4hCDnBcP2Mr71TVblfDY0wEdAEbGEJqHR1SxvWyn0UU1ZL4vTcylB/KJuV2gMhznOjbnQ6cjAhr2JYpweTYzz3wIDAQAB" )  ; ----- DKIM key vps.gio.user for debian.org
Then you have to carefully copy the content of the p= field (without being fooled by it being split between different strings) and construct a request of the form:
dkimPubKey: vps.gio.user MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsM/W/kxtKWT58Eak0cfm/ntvurfbkkvugrG2jfvSMnHHkFyfJ34Xvn/HhQPLwX1QsjhuLV+tW+BQtxY7jxSABCee6nHQRBrpDej1t86ubw3CSrxcg1mzJI5BbL8un0cwYoBtUvhCYAZKarv1W2otCGs43L0sGtEqqtmYN/hIVVm4FcqeYS1cYrZxDsjPzCEocpYBhqHh1MTeUEddVmPHKZswzvllaWF0mgIXrfDNAE0LiX39aFKWtgvflrYFKiL4hCDnBcP2Mr71TVblfDY0wEdAEbGEJqHR1SxvWyn0UU1ZL4vTcylB/KJuV2gMhznOjbnQ6cjAhr2JYpweTYzz3wIDAQAB
and then send it GPG-signed to changes@db.debian.org:
echo 'dkimPubKey: vps.gio.user blahblahblah'   gpg --clearsign   mail changes@db.debian.org
Then use host -t TXT vps.gio.user._domainkey.debian.org to chech the key gets published (it will probably take some minutes/hours, I don't know). Once it is published, you can enable DKIM in you mail server and your email will be signed. Congratulations, you will not look like a spammer any more! You can send an email to check-auth@verifier.port25.com to check that your setup is correct. They will reply with a report, including the success of DKIM test. Notice that currently Debian's setup only allows you to use RSA DKIM keys and doesn't allow you to set other DKIM fields (but you probably won't need to set them). EDIT DSA made an official announcement about DKIM support, which you might want to check out as well, together with its links. EDIT 2 Now ed25519 keys are supported, the syntax for specifying keys on LDAP is a little bit more flexible and you can also insert CNAME records. See the official documentation for the updated details. So we have solved our problems with spam? Ha, no! DKIM is only a small step. Useful, also because it enable other steps to be taken in the future, but small. In particular, DKIM enables you to say: "This particular email actually comes from @debian.org", but doesn't tell anybody what to do with emails that are not signed. A third-party mail server might wonder whether @debian.org emails are actually supposed to be signed or not. There is another standard for dealing with that, which is called DMARD, and I believe that Debian should eventually use it, but not now: the problem is that currently virtually no email from @debian.org is signed with DKIM, so if DMARC was enabled other mail servers would start to nuke all @debian.org emails, except those which are already signed, a minority. If people and services sending emails from @debian.org will start configuring DKIM on their servers, which is now possible, it will eventually come a time when DMARC can be enabled, and spammers will find themselves unable to send forged @debian.org emails. We are not there yet, but todays we are a little step closer than yesterday. Also, notice that having DKIM on @debian.org only counters spam pretending to be from @debian.org, but there is much more. The policy on what to accept is mostly independent on that on what you send. However, knowing that @debian.org emails have DKIM and DMARC would mean that we can set our spam filters to be more aggressive in general, but whitelist official Debian Developers and services. And the same can be done for other domains using DKIM and DMARC. Finally, notice that some incompatibilities between DKIM and mailing lists are known, and do not have a definitive answer yet. Basically, most mailing list engines modify either the body of the headers in forwarded emails, which means that DKIM does not validate any more. There are many proposed solutions, possibly none completely satisfying, but since spam is not very satisfying as well, something will have to be worked out. I wrote a lot already, though, so I wont't discuss this here.

14 November 2016

Dimitri John Ledkov: /boot less LVM rootfs in Zesty


On Ubuntu many of the default boot loaders support booting kernels located on LVM volumes. This includes following platforms

  • i686, x86_64 bios grub2
  • arm64, armhf, i686, x86_64 UEFI grub2
  • PReP partitions on IBM PowerPC
  • zipl on IBM zSystems
For all of the above the d-i has been modified in Zesty to create LVM based installations without a dedicated /boot partition. We shall celebrate this achievement. Hopefully this means one doesn't need to remove kernels as much, or care about sizing /boot volume appropriately any more.

If there are more bootloaders in Ubuntu that support booting off LVM, please do get in touch with me. I'm interested if I can safely enable following platforms as well:
  • armhf with u-boot
  • arm64 with u-boot
  • ppc64el with PReP volume
ps. boots pic is from here

18 October 2016

MJ Ray: Rinse and repeat

Forgive me, reader, for I have sinned. It has been over a year since my last blog post. Life got busy. Paid work. Another round of challenges managing my chronic illness. Cycle campaigning. Fun bike rides. Friends. Family. Travels. Other social media to stroke. I m still reading some of the planets where this blog post should appear and commenting on some, so I ve not felt completely cut off, but I am surprised how many people don t allow comments on their blogs any more (or make it too difficult for me with reCaptcha and the like). The main motive for this post is to test some minor upgrades, though. Hi everyone. How s it going with you? I ll probably keep posting short updates in the future. Go in peace to love and serve the web.

11 June 2015

MJ Ray: Mick Morgan: here s why pay twice?

http://baldric.net/2015/06/05/why-pay-twice/ asks why the government hires civilians to monitor social media instead of just giving GC HQ the keywords. Us cripples aren t allowed to comment there (physical ability test) so I reply here: It s pretty obvious that they have probably done both, isn t it? This way, they re verifying each other. Politicians probably trust neither civilians or spies completely and that makes it worth paying twice for this. Unlike lots of things that they seem to want not to pay for at all

14 May 2015

MJ Ray: Recorrecting Past Mistakes: Window Borders and Edges

A while ago, I switched from tritium to herbstluftwm. In general, it s been a good move, benefitting from active development and greater stability, even if I do slightly mourn the move from python scripting to a shell client. One thing that was annoying me was that throwing the pointer into an edge didn t find anything clickable. Window borders may be pretty, but they re a pretty poor choice as the thing that you can locate most easily, the thing that is on the screen edge. It finally annoyed me enough to find the culprit. The .config/herbstluftwm/autostart file said hc pad 0 26 (to keep enough space for the panel at the top edge) and changing that to hc pad 0 -8 -7 26 -7 and reconfiguring the panel to be on the bottom (where fewer windows have useful controls) means that throwing the pointer at the top or the sides now usually finds something useful like a scrollbar or a menu. I wonder if this is a useful enough improvement that I should report it as an enhancement bug.

20 February 2015

MJ Ray: Rebooting democracy? The case for a citizens constitutional convention.

I m getting increasingly cynical about our largest organisations and their voting-centred approach to democracy. You vote once, for people rather than programmes, then you re meant to leave them to it for up to three years until they stand for reelection and in most systems, their actions aren t compared with what they said they d do in any way. I have this concern about Cooperatives UK too, but then its CEO publishes http://www.uk.coop/blog/ed-mayo/2015-02-18/rebooting-democracy-case-citizens-constitutional-convention and I think there may be hope for it yet. Well worth a read if you want to organise better groups.

22 January 2015

MJ Ray: Outsourcing email to Google means SPF allows phishing?

I expect this is obvious to many people but bahumbug To Phish, or Not to Phish? just woke me up to the fact that if Google hosts your company email then its Sender Policy Framework might make other Google-sent emails look legitimate for your domain. When combined with the unsupportive support of the big free webmail hosts, is this another black mark against SPF?

8 January 2015

MJ Ray: Social Network Wishlist

All I want for 2015 is a Free/Open Source Software social network which is: Is this too much to ask for? Does it exist already?

30 December 2014

MJ Ray: GPG Transition Statement

Rather late but I guess that just confirms it s really me, right? ;-) The signed text and IDs should be at http://mjr.towers.org.uk/transition-statement.txt Thank you if you help me out here :-) I ll resign keys in a while.

4 December 2014

MJ Ray: Autumn Statement #AS2014, the Google tax and how it relates to Free Software

One of the attention-grabbing measures in the Autumn Statement by Chancellor George Osborne was the google tax on profits going offshore, which may prove unworkable (The Independent). This is interesting because a common mechanism for moving the profits around is so-called transfer pricing, where the business in one country pays an inflated price to its sibling in another country for some supplies. It sounds like the intended way to deal with that is by inspecting company accounts and assessing the underlying profits. So what s this got to do with Free Software? Well, one thing the company might buy from itself is a licence to use some branding, paying a fee for reachuse. The main reason this is possible is because copyright is usually a monopoly, so there is no supplier of a replacement product, which makes it hard to assess how much the price has been inflated. One possible method of assessing the overpayment would be to compare with how much other businesses pay for their branding licences. It would be interesting if Revenue and Customs decide that there s lots of Royalty Free licensing out there including Free Software and so all licence fees paid to related companies are a tax avoidance ruse. Similarly, any premium for a particular self-branded product over a generic equivalent could be classed as profit transfer. This could have amusing implications for proprietary software producers who sell to sister companies but I doubt that the government will be that radical, so we ll continue to see absurdities like Starbucks buying all their coffee from famous coffee producing countries Switzerland and the Netherlands. Shouldn t this be stopped, really?

22 July 2014

MJ Ray: Three systems

There are three basic systems: The first is slick and easy to use, but fiddly to set up correctly and if you want to do something that its makers don t want you to, it s rather difficult. If it breaks, then fixing it is also fiddly, if not impossible and requiring complete reinitialisation. The second system is an older approach, tried and tested, but fell out of fashion with the rise of the first and very rarely comes preinstalled on new machines. Many recent installations can be switched to and from the first system at the flick of a switch if wanted. It needs a bit more thought to operate but not much and it s still pretty obvious and intuitive. You can do all sorts of customisations and it s usually safe to mix and match parts. It s debatable whether it is more efficient than the first or not. The third system is a similar approach to the other two, but simplified in some ways and all the ugly parts are hidden away inside neat packaging. These days you can maintain and customise it yourself without much more difficulty than the other systems, but the basic hardware still attracts a price premium. In theory, it s less efficient than the other types, but in practice it s easier to maintain so doesn t lose much efficiency. Some support companies for the other types won t touch it while others will only work with it. So that s the three types of bicycle gears: indexed, friction and hub. What did you think it was?

27 June 2014

MJ Ray: #coops14 sees last days of Downham Food Co-op

image While cooperatives fortnight is mostly a celebration of how well cooperatives are doing in the UK, this year is tinged with sadness for me because it sees Downham Food Coop stop trading. This Friday and Saturday will be their last market stall, 9til 1 on the Town Square, aka Clock or Pump square. As you can see, the downturn has hit the market hard and I guess being the last stall left outside the market square (see picture: it used to have neighbouring stalls!) was just too much. The coop cites shortage of volunteers and trading downturn as reasons for closure. But if you re near Downham today or tomorrow morning, please take advantage of this last chance to buy some great products in West Norfolk!

25 June 2014

MJ Ray: New comments methods

After years of resisting it, I ve added the least evil Twitter/Facebook comments plugin I could find to this blog as a test and updated the comments policy a little. Please kick the tyres and try commenting to see if it works, phase.

1 May 2014

MJ Ray: The Kelly Lessons in About 1555 Characters

So the Kelly report of the independent review into the events leading to the Co-operative Bank s capital shortfall was published yesterday. During the day, I was putting odd bits from it out in 140 characters with the hashtags #coops #kellylessons. Here they are in one more permanent place. How many of these lessons has your organisation whether a co-op or not learned?
  1. Running a full-service bank is a complex business Bank failed to understand the limits of its own capability
  2. The most important task for any board is to put in place the right Executive leadership for the business
  3. Ownership of a regulated bank requires a clearly articulated statement addressing mgmt & gov nance relationship
  4. Failures in board oversight are inevitable if the criteria used to elect do not require the necessary skills
  5. A bank board must include sufficient numbers of technically competent directors
  6. Boards need good m gmt info and to demand it if it is not forthcoming. Failure to obtain explains failings
  7. A bank should develop&implement robust risk gov nce&oversight and an appropriate control framework
  8. IT transformation keep as simple as poss , phase delivery.., deploy the right resources, plan for contingencies
  9. Bank should have paid closer attention and responded with greater urgency to what the Regulator told it
  10. Pay careful attention to the advice of external advisors. The Group ignored well-founded inconvenient advice
  11. Postponing dealing with problems is almost never a sustainable solution.
  12. Values need to be translated into meaningful guidance The Bank s ethical positioning should be more apparent
  13. Mantras about scale and ethics are no substitute for strategies grounded in a real understanding
  14. Talent management is critical Lack of capability driven by weaknesses in its recruitment&subsequent m gmt
  15. Tolerating culture of underperformance, weak transparency and a lack of accountability, constrains an organisation
Are there other lessons that you would add?

9 January 2014

MJ Ray: Request for West Norfolk to Complete the PCC Consultation

Please excuse the intrusion to your usual software and co-op news items but vine seems broken and as part of my community and democratic interests, I d like to share this short clip quoting Norfolk s Deputy Police Commissioner Jenny McKibben about why Commissioner Stephen Bett believes it s important to get views from the west of the county about next year s police budget: Personally, with a King s Lynn + West Norfolk Bike Users Group hat on, I d like it if people supported a 2% ( 4/year average) tax increase to reduce the police s funding cut (the grant from gov.uk is being cut by 4%) so that we re less likely to have future cuts to traffic policing. The consultation details and response form are on the PCC website.

9 December 2013

MJ Ray: About Co-ops & Governance

There have been some dark days for UK coops recently the crystal Methodist and all that and I have not been able to talk about it much because of the amount of work that I want to do before the end of the year. Happily good colleagues have been writing about it and here s another good article from Kate Whittle that links to Ed Mayo and Ian Snaith who are the other two that I d suggest. http://www.cooperantics.coop/2013/12/09/co-ops-governance/ I should be back in a few days to summarise the event I attended last week.

13 October 2013

Robert Collins: key transition time

I ve transitioned to a new key announcement here or below. If you ve signed my key in the past please consider signing my new key to get it integrated into the web of trust. Thanks!
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1,SHA256
Sun, 2013-10-13
Time for me to migrate to a new key (shockingly late - sorry!).
My old key is set to expire early next year. Please use my new key effective
immediately. If you have signed my old key then please sign my key - this
message is signed by both keys (and the new key is signed by my old key).
old key:
pub 1024D/FBD3EB8E 2002-07-20
Key fingerprint = 9222 8732 859D 25CC 2560 B617 867B F9A9 FBD3 EB8E
new key:
pub 4096R/AAC0E286 2013-10-13
Key fingerprint = 8244 0CEA B440 83C7 9431 D2CC 298E 9A19 AAC0 E286
The new key is up on the keyservers, so you can just pull it from there.
- -Rob
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
iEYEARECAAYFAlJZ8FEACgkQhnv5qfvT644WxACfWBoKdVW+YDrMR1H9IY6iJUk8
ZC8AoIMRc55CTXsyn3S7GWCfOR1QONVhiQEcBAEBCAAGBQJSWfBRAAoJEInv1Yjp
ddbfbvgIAKDsvPLQil/94l7A3Y4h4CME95qVT+m9C+/mR642u8gERJ1NhpqGzR8z
fNo8X3TChWyFOaH/rYV+bOyaytC95k13omjR9HmLJPi/l4lnDiy/vopMuJaDrqF4
4IS7DTQsb8dAkCVMb7vgSaAbh+tGmnHphLNnuJngJ2McOs6gCrg3Rb89DzVywFtC
Hu9t6Sv9b0UAgfc66ftqpK71FSo9bLQ4vGrDPsAhJpXb83kOQHLXuwUuWs9vtJ62
Mikb0kzAjlQYPwNx6UNpQaILZ1MYLa3JXjataAsTqcKtbxcyKgLQOrZy55ZYoZO5
+qdZ1+wiD3+usr/GFDUX9KiM/f6N+Xo=
=EVi2
-----END PGP SIGNATURE-----

3 October 2013

MJ Ray: The 2012 Update to the Worker Co-op Code

We like guidelines. In our work, things like the Debian Free Software Guidelines, pep8 and Koha Coding Guidelines are quite useful. I follow guidelines for how I work, too. In addition to the financial reports required by government, our co-op produces an annual social report which we share with our members and other key stakeholders. Since 2007, the backbone of it is The Worker Co operative Code of Governance published by our national federation. In 2012, the Worker Co-op Council updated the code. I don t remember why an update was felt necessary, but as a side-effect of producing of our 2013 social report, I ve made a list of the changes:
  1. Principle 1 is reordered, with information becoming the first point and membership offered to all becoming the last item.
  2. Principle 2 sees democratic processes drop down the order, plus it loses the item on long-term planning.
  3. Principle 3 has the point about reserves clarified and gains a last item about distributing surplus fairly.
  4. Principle 4 loses its first item about regular reviews, the skills assessment point moves to principle 5 and it gains a build capability point.
  5. Principle 5 gains items on replacing key members and skills assessment (from the previous section), while most points seem rephrased.
  6. Principle 6 is reordered, active co-operation is split into distinct points about referring and collaborating and the point about actively sharing good practice is deleted.
  7. Principle 7 is unchanged.
Are these good changes? Much of it seems like tinkering and maybe shifting emphasis the reorderings add little and make it harder to spot the changes while the lost points on long-term planning and sharing good practice are surprising. I would have preferred to see the items that seem mainly to promote the code itself and its publisher Co-operatives UK deleted instead. The additions and clarifications about surplus are good, though, and there s nothing new that I think should stop us adopting it. What do you think? Should all business behave this way?

29 September 2013

MJ Ray: Send updates to live cycling news reports to the speech synth

The BBC coverage of the UCI Women Road Race World Championship wasn t starting until 3pm, BBC Radio 5 Live had football and Sports Extra was playing an advert loop (really BBC?), Eurosport wasn t covering the race at all, RAI Sport 2 had coverage which was fine while I was watching the TV, but my Italian isn t good enough to follow the commentary and I wanted to get some other stuff done. So the obvious thing is to have the computer watch for changes to the great http://live.cyclingnews.com/ ticker and read them out, right? Well, it was to me. Here s the script I used:
#!/usr/bin/rc
# Send updates to live cycling news reports to the speech synth
# initialise
url=http://live.cyclingnews.com/
if (~ $1 *:*)  
  url=$1
 
last=""
while (true)  
    # download the update and strip html - if you don't have html2text try
    # sed -e '1,/<ol/d;s/<br[^>]*>/n/g;s/<[^>]*>//g;s/ / /g;s/[[:space:]]*$//'
    next= () curl -s $url   html2text -width 9999   sed -e 's/^ *[0-9]*. //'  
    # look for some added lines in the update section - timezone will need changing for non-European races
    say= () diff -u < echo $last  < echo $next  
        sed -ne '/^+/!d;s///;s/**//g;/ CES*T/,/^ *$/p'   head -9  
    # feed them to the speech synth in reverse order
    echo $say '
      Race Update'   tac   spd-say -e
    # update variables, pause and loop
    last=$next
    # set this to a prime number over 60, like 61, 67, 71, 73, 79, 83, 89
    sleep 67
 
Any questions, comments or improvements? I could have done more, like not saying Race Update even when there was no update, but I wanted to start listening as soon as possible!

Next.