This is part of a series of posts on compiling a custom version of Qt5 in order
to develop for both amd64 and a Raspberry Pi.
Building Qt5 takes a long time. The build server I was using had CPUs and RAM,
but was very slow on I/O. I was very frustrated by that, and I started
evaluating alternatives. I ended up setting up scripts to automatically
provision a throwaway cloud server at
Hetzner.
Initial setup
I got an API key from my customer's Hetzner account.
I installed
hcloud-cli
, currently only in testing and unstable:
Then I configured
hcloud
with the API key:
Spin up
I wrote a quick and dirty script to spin up a new machine, which grew a bit
with little tweaks:
#!/bin/sh
# Create the server
hcloud server create --name buildqt --ssh-key --start-after-create \
--type cpx51 --image debian-10 --datacenter
# Query server IP
IP="$(hcloud server describe buildqt -o json jq -r .public_net.ipv4.ip)"
# Update ansible host file
echo "buildqt ansible_user=root ansible_host=$IP" > hosts
# Remove old host key
ssh-keygen -f ~/.ssh/known_hosts -R "$IP"
# Update login script
echo "#!/bin/sh" > login
echo "ssh root@$IP" >> login
chmod 0755 login
I picked a datacenter in the same location as where we have other servers, to
get quicker data transfers.
I like that CLI tools have JSON output that I can cleanly pick at with
jq. Sadly, my ISP doesn't do IPv6 yet.
Since the server just got regenerated, I remove a possibly cached host key.
Provisioning the machine
One git server I need is behind HTTP authentication. Here's a quick hack to
pass the relevant
.netrc
credentials to ansible before provisioning:
#!/usr/bin/python3
import subprocess
import netrc
import tempfile
import json
login, account, password = netrc.netrc().authenticators(" ")
with tempfile.NamedTemporaryFile(mode="wt", suffix=".json") as fd:
json.dump(
"repo_user": login,
"repo_password": password,
, fd)
fd.flush()
subprocess.run([
"ansible-playbook",
"-i", "hosts",
"-l", "buildqt",
"--extra-vars", f"@ fd.name ",
"provision.yml",
], check=True)
And here's the ansible playbook:
#!/usr/bin/env ansible-playbook
- name: Install and configure buildqt
hosts: all
tasks:
- name: Update apt cache
apt:
update_cache: yes
cache_valid_time: 86400
- name: Create build user
user:
name: build
comment: QT5 Build User
shell: /bin/bash
- name: Create sources directory
become: yes
become_user: build
file:
path: ~/sources
state: directory
mode: 0755
- name: Download sources
become: yes
become_user: build
get_url:
url: "https:// / item "
dest: "~/sources/ item "
mode: 0644
with_items:
- "qt-everywhere-src-5.15.1.tar.xz"
- "qt-creator-enterprise-src-4.13.2.tar.gz"
- name: Populate home directory
become: yes
become_user: build
copy:
src: build
dest: ~/
mode: preserve
- name: Write .netrc
become: yes
become_user: build
copy:
dest: ~/.netrc
mode: 0600
content:
machine
login repo_user
password repo_password
- name: Write .screenrc
become: yes
become_user: build
copy:
dest: ~/.screenrc
mode: 0644
content:
hardstatus alwayslastline
hardstatus string '% = cw %-Lw% = KW %50>%n%f* %t% = cw %+Lw%< % = kK %-=%D %Y-%m-%d %c% - '
startup_message off
defutf8 on
defscrollback 10240
- name: Install base packages
apt:
name: git,mc,ncdu,neovim,eatmydata,devscripts,equivs,screen
state: present
- name: Clone git repo
become: yes
become_user: build
git:
repo: https:// @ / .git
dest: ~/
- name: Copy Qt license
become: yes
become_user: build
copy:
src: qt-license.txt
dest: ~/.qt-license
mode: 0600
Now everything is ready for a 16 core, 32Gb ram build on SSD storage.
Tear down
When done:
#!/bin/sh
hcloud server delete buildqt
The whole spin up plus provisioning takes around a minute, so I can do it when
I start a work day, and take it down at the end. The build machine wasn't that
expensive to begin with, and this way it will even be billed by the hour.
A first try on a CPX51 machine has just built the full Qt5 Everywhere
Enterprise including QtWebEngine and all its frills, for
amd64
, in under 1
hour and 40 minutes.