Search Results: "Brett Parker"

1 March 2017

Brett Parker: Using the Mythic Beasts IPv4 -> IPv6 Proxy for Websites on a v6 only Pi and getting the right REMOTE_ADDR

So, more because I was intrigued than anything else, I've got a pi3 from Mythic Beasts, they're supplied with IPv6 only connectivity and the file storage is NFS over a private v4 network. The proxy will happily redirect requests to either http or https to the Pi, but this results (without turning on the Proxy Protocol) with getting remote addresses in your logs of the proxy servers, which is not entirely useful. I've cheated a bit, because the turning on of ProxyProtocol for the hostedpi.com addresses is currently not exposed to customers (it's on the list!), to do it without access to Mythic's backends use your own domainname (I've also got https://pi3.sommitrealweird.co.uk/ mapped to this Pi). So, first step first, we get our RPi and we make sure that we can login to it via ssh (I'm nearly always on a v6 connection anyways, so this was a simple case of sshing to the v6 address of the Pi). I then installed haproxy and apache2 on the Pi and went about configuring them, with apache2 I changed it to listen to localhost only and on ports 8080 and 4443, I hadn't at this point enabled the ssl module so, really, the change for 4443 didn't kick in. Here's my /etc/apache2/ports.conf file:
# If you just change the port or add more ports here, you will likely also
# have to change the VirtualHost statement in
# /etc/apache2/sites-enabled/000-default.conf
Listen [::1]:8080
<IfModule ssl_module>
       Listen [::1]:4443
</IfModule>
<IfModule mod_gnutls.c>
       Listen [::1]:4443
</IfModule>
# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
I then edited /etc/apache2/sites-available/000-default.conf to change the VirtualHost line to [::1]:8080. So, with that in place, now we deploy haproxy infront of it, the basic /etc/haproxy/haproxy.cfg config is:
global
       log /dev/log    local0
       log /dev/log    local1 notice
       chroot /var/lib/haproxy
       stats socket /run/haproxy/admin.sock mode 660 level admin
       stats timeout 30s
       user haproxy
       group haproxy
       daemon
       # Default SSL material locations
       ca-base /etc/ssl/certs
       crt-base /etc/ssl/private
       # Default ciphers to use on SSL-enabled listening sockets.
       # For more information, see ciphers(1SSL). This list is from:
       #  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
       ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
       ssl-default-bind-options no-sslv3
defaults
       log     global
       mode    http
       option  httplog
       option  dontlognull
        timeout connect 5000
        timeout client  50000
        timeout server  50000
       errorfile 400 /etc/haproxy/errors/400.http
       errorfile 403 /etc/haproxy/errors/403.http
       errorfile 408 /etc/haproxy/errors/408.http
       errorfile 500 /etc/haproxy/errors/500.http
       errorfile 502 /etc/haproxy/errors/502.http
       errorfile 503 /etc/haproxy/errors/503.http
       errorfile 504 /etc/haproxy/errors/504.http
frontend any_http
        option httplog
        option forwardfor
        acl is_from_proxy src 2a00:1098:0:82:1000:3b:1:1 2a00:1098:0:80:1000:3b:1:1
        tcp-request connection expect-proxy layer4 if is_from_proxy
        bind :::80
        default_backend any_http
backend any_http
        server apache2 ::1:8080
Obviously after that you then do:
systemctl restart apache2
systemctl restart haproxy
Now you have a proxy protocol'd setup from the proxy servers, and you can still talk directly to the Pi over ipv6, you're not yet logging the right remote ips, but we're a step closer. Next enable mod_remoteip in apache2:
a2enmod remoteip
And add a file, /etc/apache2/conf-available/remoteip-logformats.conf containing:
LogFormat "%v:%p %a %l %u %t \"%r\" %>s %O \"% Referer i\" \"% User-Agent i\"" remoteip_vhost_combined
And edit the /etc/apache2/sites-available/000-default.conf to change the CustomLog line to use remoteip_vhost_combined rather than combined as the LogFormat and add the relevant RemoteIP settings:
RemoteIPHeader X-Forwarded-For
RemoteIPTrustedProxy ::1
CustomLog $ APACHE_LOG_DIR /access.log remoteip_vhost_combined
Now, enable the config and restart apache2:
a2enconf remoteip-logformats
systemctl restart apache2
Now you'll get the right remote ip in the logs (cool, huh!), and, better still, the environment that gets pushed through to cgi scripts/php/whatever is now also correct. So, you can now happily visit http://www.<your-pi-name>.hostedpi.com/, e.g. http://www.srwpi.hostedpi.com/. Next up, you'll want something like dehydrated - I grabbed the packaged version from debian's jessie-backports repository - so that you can make yourself some nice shiny SSL certificates (why wouldn't you, after all!), once you've got dehydrated installed, you'll probably want to tweak it a bit, I have some magic extra files that I use, I also suggest getting the dehydrated-apache2 package, which just makes it all much easier too. /etc/dehydrated/conf.d/mail.sh:
CONTACT_EMAIL="my@email.address"
/etc/dehydrated/conf.d/domainconfig.sh:
DOMAINS_D="/etc/dehydrated/domains.d"
/etc/dehydrated/domains.d/srwpi.hostedpi.com:
HOOK="/etc/dehydrated/hooks/srwpi"
/etc/dehydrated/hooks/srwpi:
#!/bin/sh
action="$1"
domain="$2"
case $action in
  deploy_cert)
    privkey="$3"
    cert="$4"
    fullchain="$5"
    chain="$6"
    cat "$privkey" "$fullchain" > /etc/ssl/private/srwpi.pem
    chmod 640 /etc/ssl/private/srwpi.pem
    ;;
  *)
    ;;
esac
/etc/dehydrated/hooks/srwpi has the execute bit set (chmod +x /etc/dehydrated/hooks/srwpi), and is really only there so that the certificate can be used easily in haproxy. And finally the file /etc/dehydrated/domains.txt:
www.srwpi.hostedpi.com srwpi.hostedpi.com
Obviously, use your own pi name in there, or better yet, one of your own domain names that you've mapped to the proxies. Run dehydrated in cron mode (it's noisy, but meh...):
dehydrated -c
That s then generated you some shiny certificates (hopefully). For now, I'll just tell you how to do it through the /etc/apache2/sites-available/default-ssl.conf file, just edit that file and change the SSLCertificateFile and SSLCertificateKeyFile to point to /var/lib/dehydrated/certs/www.srwpi.hostedpi.com/fullchain.pem and /var/llib/dehydrated/certs/ww.srwpi.hostedpi.com/privkey.pem files, do the edit for the CustomLog as you did for the other default site, and change the VirtualHost to be [::1]:443 and enable the site:
a2ensite default-ssl
a2enmod ssl
And restart apache2:
systemctl restart apache2
Now time to add some bits to haproxy.cfg, usefully this is only a tiny tiny bit of extra config:
frontend any_https
        option httplog
        option forwardfor
        acl is_from_proxy src 2a00:1098:0:82:1000:3b:1:1 2a00:1098:0:80:1000:3b:1:1
        tcp-request connection expect-proxy layer4 if is_from_proxy
        bind :::443 ssl crt /etc/ssl/private/srwpi.pem
        default_backend any_https
backend any_https
        server apache2 ::1:4443 ssl ca-file /etc/ssl/certs/ca-certificates.crt
Restart haproxy:
systemctl restart haproxy
And we're all done! REMOTE_ADDR will appear as the correct remote address in the logs, and in the environment.

Brett Parker: Ooooooh! Shiny!

Yay! So, it's a year and a bit on from the last post (eeep!), and we get the news of the Psion Gemini - I wants one, that looks nice and shiny and just the right size to not be inconvenient to lug around all the time, and far better for ssh usage than the onscreen keyboard on my phone!

28 June 2014

Brett Parker: Sony Entertainment Networks Insanity

So, I have a SEN account (it's part of the PSN), I have 2 videos with SEN, I have a broken PS3 so I can no deactivate video (you can only do that from the console itself, yes, really)... and the response from SEN has been abysmal, specifically:
As we take the security of SEN accounts very seriously, we are unable to provide support on this matter by e-mail as we will need you to answer some security questions before we can investigate this further. We need you to phone us in order to verify your account details because we're not allowed to verify details via e-mail.
I mean, seriously, they're going to verify my details over the phone better than over e-mail how exactly? All the contact details are tied to my e-mail account, I have logged in to their control panel and renamed the broken PS3 to "Broken PS3", I have given them the serial number of the PS3, and yet they insist that I need to call them, because apparently they're fucking stupid. I'm damned glad that I only ever got 2 videos from SEN, both of which I own on DVD now anyways, this kind of idiotic tie in to a system is badly wrong. So, you phone the number... and now you get stuck with hold music for ever... oh, yeah, great customer service here guys. I mean, seriously, WTF. OK - 10 minutes on the phone, and still being told "One of our advisors will be with you shortly". I get the feeling that I'll just be writing off the 2 videos that I no longer have access to. I'm damned glad that I didn't decide to buy more content from that - at least you can reset the games entitlement once every six months without jumping through all these hoops (you have to reactivate each console that you still want to use, but hey).

4 January 2014

Brett Parker: Wow, I do believe Fasthosts have outdone themselves...

So, got a beep this morning from our work monitoring system. One of our customers domain names is hosted with livedns.co.uk (which, as far as I can tell, is part of the Fasthosts franchise)... It appears that Fasthosts have managed to entirely break their DNS:
brettp@laptop:~$ host www.fasthosts.com
;; connection timed out; no servers could be reached
brettp@laptop:~$ whois fasthosts.com   grep -i "Name Server"
   Name Server: NS1.FASTHOSTS.NET.UK
   Name Server: NS2.FASTHOSTS.NET.UK
Name Server: NS1.FASTHOSTS.NET.UK
Name Server: NS2.FASTHOSTS.NET.UK
brettp@laptop:~$ whois fasthosts.net.uk   grep -A 2 "Name servers:"
    Name servers:
        ns1.fasthosts.net.uk      213.171.192.252
        ns2.fasthosts.net.uk      213.171.193.248
brettp@laptop:~$  host -t ns fasthosts.net.uk 213.171.192.252
;; connection timed out; no servers could be reached
brettp@laptop:~$ host -t ns fasthosts.net.uk 213.171.193.248
;; connection timed out; no servers could be reached
brettp@laptop:~$
So, that's fasthosts core nameservers not responding, good start! They also provide livedns.co.uk, so lets have a look at that:
brettp@laptop:~$ whois livedns.co.uk   grep -A 3 "Name servers:"
    Name servers:
        ns1.livedns.co.uk         213.171.192.250
        ns2.livedns.co.uk         213.171.193.250
        ns3.livedns.co.uk         213.171.192.254
brettp@laptop:~$ host -t ns ns1.livedns.co.uk 213.171.192.250
;; connection timed out; no servers could be reached
brettp@laptop:~$ host -t ns ns1.livedns.co.uk 213.171.193.250
;; connection timed out; no servers could be reached
brettp@laptop:~$ host -t ns ns1.livedns.co.uk 213.171.192.254
;; connection timed out; no servers could be reached
So, erm, apparently that's all their DNS servers "Not entirely functioning correctly"! That's quite impressive!

3 December 2013

Brett Parker: dd over ssh oddness

So, using the command:
root@new# ssh root@old dd if=/dev/vg/somedisk dd of=/dev/vg/somedisk
appears to fail, getting a SIGTERM at some point for no discernable reason... however, using
root@old# dd if=/dev/vg/somedisk ssh root@new dd of=/dev/vg/somedisk
works fine. The pull version fails at a fairly random point after a fairly undefined period of time. The push version works everytime. This is most confusing and odd... Dear lazyweb, please give me some new ideas as to what's going on, it's driving me nuts! Update: solved... A different daemon wasn't limiting it's killing habits in the case that a certain process wasn't running, and was killing the ssh process on the new server almost at random, found the bug in the code and now testing with that. Thanks for all the suggestions though, much appreciated.

16 October 2013

Brett Parker: Hepworth Spam

There's been a recent spate of spam from domains held by Communicado Ltd, to counter this Martin A Brooks has created a lovely list of domains to easily filter, see his blog post for information on the distribution and usage of this list.

14 October 2013

Brett Parker: And on a different but similar tack...

Of the (entire!) 35 comments on my blog that I've approved, 5 of those are actually from ipv6 addresses! Well done you ipv6 early uptakers! You all deserve a pint. (If you ever happen to turn up in the same pub as me, you should probably remind me I said that :) )

Brett Parker: Just in case you missed it...

So, I've changed jobs, I now work for the incredibly awesome Mythic Beasts, So far it's been great, we've hit a few problems, we've made sane plans, and we're moving forwards and making things easier to manage long term. I've mostly been involved with debugging mac-mini boot problems, and getting them working with (as far as possible) stock kernels, rather than our previous set of custom compiled kernels, so that it's easier to upgrade later, and so that we're not having to maintain a whole set of patches that we shouldn't need to! We're also starting to package our own software that we use internally so that actually deploying new servers is becoming a lot easier, and configuration of our previous packages is now (mostly) handled by debconf, meaning that actually setting up backups is becoming much easier. We've also changed out main backup strategy from using a (slightly flaky) openvpn connection with loopback mounted ext3 filesystems to using rsync with the --fake-super option - the only thing that we've found slightly wrong with this so far is that --fake-super can't reconcile symlinks (it uses user level xattr to store the origional permissions, and with symlinks this doesn't work, because, apparently, symlinks should store user xattr attrbutes!). We also offer ipv6 for all our dedicated and vps customers, as per the mythic blog - and our 2 authoritative name servers are both v6 enabled. We're working towards making all of our core services available over both ipv4 and ipv6, and we should be there soon. It's fantastic to work for a company that has the technical knowledge and no-shit stance that Mythic Beasts has, we will hold our hands up if we do something wrong, and we'll tell you exactly what happened, why, and what we're doing to stop it from happening again. So, 1 month in, and I'm absolutely loving working for them, and I've been a customer of theirs since black cat networks sold their domain name side to them, and shortly afterwards bluelinux sold their virtual servers service to them. Our new VDS lite service is reasonably priced, and well spec'd. I thoroughly recommend our services, and obviously, we actually use our own vds service to provide some of our services - so it's as important to us that it keeps working as it is for our customers. Wow, so that was a very salesy post. Erm, next time, back to whinging about authors not having written the books I'm waiting for yet! :)

Brett Parker: WHSmith - How not to do online trade...

So, apparently WHSmith are very upset with the appearance of "Unacceptable Titles" in their Kobo feed, and so, the obvious solution to this is to entirely take down their website and replace it with the following holding page /media/photos/1-website-images/whsmith-small.png So, rather than doing the sensible thing, of just removing the kobo feed temporarily, and putting a note on the ebook selling part of their website, instead they've gone with the TAKE EVERYTHING DOWN!!1!!1!1!!one! approach. That seems somewhat stupid, and I'd be looking at whoever made that call and looking to find out how they've still got a job... unless WHSmith really do absolutely no online trade at all, this seems to be particularly stupid.

18 September 2013

Brett Parker: The usefulness of getent

It appears that if you want to do dns lookups, the output of host is a pain to parse, and dig doesn't follow the systems normal search path. So, the solution is to use getent ahostsv4 <hostname> which has easier to parse output (if you're only looking for v4 addresses). Apparently this is not a well known thing, so I thought I'd drop it here to tell the world :)

11 September 2013

Brett Parker: Moving, Changing Jobs and the Bank Holiday Weekend...

So, in the last couple of weeks I've (mostly) moved out of Brighton, got a new job, and generally the world has gone crazy. The bank holiday weekend was spent at Debian UK BBQ, and was fantastic fun, including the reading corner on Sunday where we played through various arcs of To Be or Not To Be (small spoiler: PIRATES!). Since then I've been mostly back and forth from Brighton moving stuff, buying a car, and starting a new job... As mentioned in my new employers blog there was a small stumbling block with my new work laptop that stopped me from getting a sensible operating system on it from the get-go, but it's now running a shiny Debian Wheezy install and alllllllllll is well with the world... well, mostly. I have just removed gnome-keyring because I hate that I can't see a way of relocking it, or infact relocking any of the individual keys that it had. So, back to ssh-agent I went, and the world is good again. But that means that I haven't got a gpg agent running at the moment, though, unless I'm missing something, I can't see a way of forcing gpg-agent to forget a passphrase either (ARGH, why on earth wouldn't you have an option for that?!).

1 August 2013

Brett Parker: Of hosting companies that are completely opaque...

So, my current place of employ (for another 3 weeks and 1 day) have a bunch of dedicated servers from a large hosting company in the UK, occasionally they break their already truly broken internal networking (yes, it defaults to blocking nearly all TCP ports between servers in their own data center, but if you want to hit those same ports from outside their network, that's absolutely fine!), I get to deal with the same thread at least once a quarter, when something in their network decides to block one of the few ports that are allowed... that port is 22, and ssh is used "rather a lot" between the approx 50 servers we have with them. Just for shits and giggles, all of their infrastructure filters ICMP making traceroute mostly totally useless. tcptraceroute ain't no better. Repeating the same question, asking for what they did when it magically all then works again, and getting a "we didn't do anything!" response is somewhat taxing, almost as taxing as watching them run commands on the servers and them not actually know what they're doing, but appear to have a vague grasp on iptables being a firewall, and that they can maybe read the rules and go, "oh, you weren't lying.". Their support is also so far lagged that it's ridiculous, other than an auto response, it took 3 hours (during office hours) to get a "oh, yeah, what you've said, that all looks good, now give us access so that we can diagnose further..." when you do give them that... it'll be between 4 and 6 hours before they bother logging in, and closer to 8 to 10 hours for them to go "oh, it works now, but I don't know why!". NYARGH. These guys make bedroom ISPs look professional to the max. The only thing they have almost going for them is that if you order a new dedicated server it is usually spun up within the hour. And here endeth the vent.

29 July 2013

Brett Parker: Of weekends and patio laying...

So, this weekend I headed up to Uncle Steve's and assisted in the laying of the flagstones for the patio for the BBQ - there's even at least one bit of photographic evidence of me in a non-supervisory capacity. Was a good weekend, and fun was had by all, even if we couldn't really express it due to Complete Exhaustion by the end of it. Only another few weekends and it'll be the Debian UK Party 2013 which will, as it is every year, be absolutely awesome. I'll leave the thanks for the beer and food donations until after the party :)

22 July 2013

Brett Parker: Replacing Planet Planet

I'm just replacing the code running planet alug from a (very old) Planet 2.0 to Venus, appears that the transition has mostly worked, but maybe it's time to think about what to replace that with too, it'd be nice, for instance, to use something shiny and django based so that some more people can add feeds (with out me having to go edit the file!).

9 July 2013

Brett Parker: It&#39;s been a looong time since the last blog post...

So, does anyone have any clue what's happened, because I'm at a loss. Apparently I didn't blog about the loss of my Dad back in December, he was found in Union Canal on the 13th December 2012, with his dinner in a carrier bag and his bike. The 'official' date of death is the 13th December 2012, but as far as we can work out (and I'm fairly sure on it), he died between 1930 and 2030 on the 12th. I was on the phone to him at 1930, and I was trying to call him back at 2030 on leaving a pub and wandering back in to town, apparently I wasn't the only one trying to call him, but we all put it down to him being out of signal at the time, until I got a call from my brother on the 13th at just gone midday, saying that the police had been round to tell him they'd found Dad in the canal that morning. The funeral was held on the 22nd January 2013, with a large turnout of friends and family, it was a good send off for the old chap, and we made damned sure that it was an even better wake - The Brewery Tap did a fantastic job for us, and all that turned up (ok, with some exceptions, but we can block them out!) were fantastic. The boy knew more people than we could possibly imagine, and most of them turned up to either the funeral, the wake or both. Still miss him, don't expect that to change anytime soon, if only because I used to talk to him most evenings after work between the train station and the pub, and I really miss being able to talk shit with him and being able to relate. Next week we should finally have the Certificate of Representation so that we can actually empty his canal boat ready to sell, at which point it's likely to go back down to Wilton Marina to be sold off. It's not a bad little boat, but it was his sanctuary, and none of us feel comfortable keeping it. In other news, work has been slightly a bit busy since then, and I'm still playing a bit of catchup from missing fair chunks of January (though, I was mostly still working from home when I could!). I'm sure other stuff must have happened... Oh, yeah, I got older, but obviously no wiser. I went and visited The Brewery Tap for Father's Day (that was a good day, 7 hours trains for 5 hours in the pub, but well worth it). Coming up: Cambridge Visitations! The August Bank Holiday Debian Party in Cambridge - now, that I am looking forward to!

18 February 2013

Brett Parker: It&#39;s that time of year again..

It's that time of year again, my hair has all gone! Slightly beardless me

13 February 2013

Brett Parker: Public Libraries...

OK - so I've just been reading the Gaurdian Article on Terry Deary saying that Libraries are outdated and should be got rid of. I entirely disagree with him, I spent a fair amount of my youth borrowing books from my local library, when I had no income, and so the only alternative to borrowing from the library would have been saving up for second hand books... which he wouldn't have seen a penny from. These days I buy Quite A Few books (OK - so, mostly on the kindle these days, but still), if it hadn't have been for libraries when I was younger and couldn't afford to purchase books, I may never have properly picked up the reading habit. His claim that they're killing bookshops is also, in my opinion, entirely wrong. Bookshops are closing more because of the ease of ordering books online and getting them delivered to your door, with a huge collection of books available from large warehouses rather than the stock that a book store can sell easily. We've got a vast collection of literature available to us now, and it's only ever going upwards, no book shop or single library is going to be able to cater for the entirely different needs of their customers. Libraries do inter library lending, which means that the collection of rarer, less popular books are still available (potentially with a bit of a wait), and every time the book is lent the author gets some funds. If he seriously thinks that if libraries closed the number of people reading his material would stay the same I think he's mistaken. He also doesn't seem to take in to account at all the second hand book market. All I have to say is NYARGH.

8 August 2012

Brett Parker: The Weirdness of People

OK - so you go to any place that has a social gathering, pub, club, bbq, anything - why is it that what invariably happens is that everyone manages to gather in the places that are most inconvienient?! E.g. in a pub, how is it that everyone gathers around the point that makes getting to the toilet difficult? Or outside? Why is it that at house type parties/bbqs everyone manages to end up in the kitchen?! (OK - I know that I'll probably be guilty of the kitchen thing, but meh!) - There must be something built in to us to end up gathering in the places that everyone is going to need to go sooner or later, are we really that social? Or is this just a need to be recognised? I don't really know... just that it's very annoying trying to get from the bar stool in the pub (where reading a book) to the toilet was a lot harder than it should have been!

24 June 2012

Brett Parker: Mira Grant - Countdown

I've really got to get a better method down of watching for books than my current periodic searching and checking and hoping, but... Hoorah - Countdown release date has been done - and it's only just over a week away - YAY!

16 June 2012

Brett Parker: Things that piss me off #1

You know what really pisses me off? Idiots in takeaways... More to the point, racist, idiotic, drunken idiots in takeaways - having just been to one of the few places still open at "silly o'clock" i.e. a place that's open after the pub has shut, it's really annoying and disappointing in the human race, to hear someone refer to the staff as a "paki bastard". I'm disgusted with the customer in question. I mean, WTF, how absolutely fucking stupid have you got to be to both be racist and do it loud enough to not only piss off the staff, but piss off any customer that actually just views people as people - we're in this world together ffs, stop being such fucking moronic, racist cunts. It's not difficult. People are people. All of them. The whole world. (apologies for bad language in the post, it's just, well, as the title says, something that really pisses me off).

Next.