I ve had a kludgy mess of electronic address books for most of two decades,
and have got rather fed up with it. My stack consisted of:
~/.mutt/aliases, a flat text file consisting of mutt alias commands- lbdb configuration to query
~/.mutt/aliases, Debian s LDAP database, and Canonical s LDAP database,
so that I can search by name with Ctrl-t in mutt when composing a new message
- Google Contacts, which I used from Android and was completely separate
from all of the above
The biggest practical problem with this was that I had the address book that
was most convenient for me to add things to (Google Contacts) and the one I
used when sending email, and no sensible way to merge them or move things
between them. I also wasn t especially comfortable with having all my
contact information in a proprietary web service.
My goals for a replacement address book system were:
- free software throughout
- storage under my control
- single common database
- minimal manual transcription when consolidating existing databases
- integration with Android such that I can continue using the same
contacts, messaging, etc. apps
- integration with
mutt such that I can continue using the same query interface
- not having to write my own software, because honestly
I think I have all this now!
New stack
The obvious basic technology to use is
CardDAV: it s fairly complex,
admittedly, but lots of software supports it and one of my goals was not
having to write my own thing. This meant I needed a CardDAV server, some
way to sync the database to and from both Android and the system where I run
mutt, and whatever query glue was necessary to get
mutt to understand vCards.
There are lots of different alternatives here, and if anything the problem
was an embarrassment of choice. In the end I just decided to go for things
that looked roughly the right shape for me and tried not to spend too much
time in analysis paralysis.
CardDAV server
I went with
Xandikos for the
server, largely because I know Jelmer and have generally had pretty good
experiences with their software, but also because using Git for history of
the backend storage seems like something my future self will thank me for.
It isn t packaged in stretch, but it s in Debian unstable, so I installed it
from there.
Rather than the standalone mode suggested on the web page, I decided to set
it up in what felt like a more robust way using
WSGI. I installed
uwsgi,
uwsgi-plugin-python3, and
libapache2-mod-proxy-uwsgi, and created the
following file in
/etc/uwsgi/apps-available/xandikos.ini which I then
symlinked into
/etc/uwsgi/apps-enabled/xandikos.ini:
[uwsgi]
socket = 127.0.0.1:8801
uid = xandikos
gid = xandikos
umask = 022
master = true
cheaper = 2
processes = 4
plugin = python3
module = xandikos.wsgi:app
env = XANDIKOSPATH=/srv/xandikos/collections
The port number was arbitrary, as was the path. You need to create the
xandikos user and group first (
adduser --system --group --no-create-home
--disabled-login xandikos). I created
/srv/xandikos owned by
xandikos:xandikos and mode 0700, and I recommend
setting a
umask as shown above since uwsgi s default
umask is 000 (!). You should also run
sudo -u xandikos xandikos -d
/srv/xandikos/collections --autocreate and then Ctrl-c it after a short
time (I think it would be nicer if there were a way to
ask the WSGI wrapper
to do this).
For Apache setup, I kept it reasonably simple: I ran
a2enmod proxy_uwsgi,
used
htpasswd to create
/etc/apache2/xandikos.passwd with a username and
password for myself, added a virtual host in
/etc/apache2/sites-available/xandikos.conf, and enabled it with
a2ensite
xandikos:
<VirtualHost *:443>
ServerName xandikos.example.org
ServerAdmin me@example.org
ErrorLog /var/log/apache2/xandikos-error.log
TransferLog /var/log/apache2/xandikos-access.log
<Location />
ProxyPass "uwsgi://127.0.0.1:8801/"
AuthType Basic
AuthName "Xandikos"
AuthBasicProvider file
AuthUserFile "/etc/apache2/xandikos.passwd"
Require valid-user
</Location>
</VirtualHost>
Then
service apache2 reload, set the new virtual host up with
Let s
Encrypt, reloaded again, and off we go.
Android integration
I installed
DAVdroid from the Play Store: it
cost a few pounds, but I was
OK with that since it s GPLv3 and I m happy to
help fund free software. I created two accounts, one for my existing Google
Contacts database (and in fact calendaring as well, although I don t intend
to switch over to self-hosting that just yet), and one for the new Xandikos
instance. The
Google
setup was a bit fiddly
because I have two-step verification turned on so I had to create an
app-specific password. The Xandikos setup was straightforward: base
URL,
username, password, and done.
Since I didn t completely trust the new setup yet, I followed what seemed
like the most robust option from the
DAVdroid contacts syncing
documentation,
and used the stock contacts app to export my Google Contacts account to a
.vcf file and then import that into the appropriate DAVdroid account
(which showed up automatically). This seemed straightforward and everything
got pushed to Xandikos. There are some weird delays in syncing contacts
that I don t entirely understand, but it all seems to get there in the end.
mutt integration
First off I needed to sync the contacts. (In fact I happen to run
mutt on
the same system where I run Xandikos at the moment, but I don t want to rely
on that, and going through the CardDAV server means that I don t have to
poke holes for myself using filesystem permissions.) I used
vdirsyncer for this. In
~/.vdirsyncer/config:
[general]
status_path = "~/.vdirsyncer/status/"
[pair contacts]
a = "contacts_local"
b = "contacts_remote"
collections = ["from a", "from b"]
[storage contacts_local]
type = "filesystem"
path = "~/.contacts/"
fileext = ".vcf"
[storage contacts_remote]
type = "carddav"
url = "<Xandikos base URL>"
username = "<my username>"
password = "<my password>"
Running
vdirsyncer discover and
vdirsyncer sync then synced everything
into
~/.contacts/. I added an hourly
crontab entry to run
vdirsyncer
-v WARNING sync.
Next, I needed a command-line address book tool based on this.
khard looked about right and is in
stretch, so I installed that. In
~/.config/khard/khard.conf (this is
mostly just the example configuration, but I preferred to sort by first name
since not all my contacts have neat first/last names):
[addressbooks]
[[contacts]]
path = ~/.contacts/<UUID of my contacts collection>/
[general]
debug = no
default_action = list
editor = vim
merge_editor = vimdiff
[contact table]
# display names by first or last name: first_name / last_name
display = first_name
# group by address book: yes / no
group_by_addressbook = no
# reverse table ordering: yes / no
reverse = no
# append nicknames to name column: yes / no
show_nicknames = no
# show uid table column: yes / no
show_uids = yes
# sort by first or last name: first_name / last_name
sort = first_name
[vcard]
# extend contacts with your own private objects
# these objects are stored with a leading "X-" before the object name in the vcard files
# every object label may only contain letters, digits and the - character
# example:
# private_objects = Jabber, Skype, Twitter
private_objects = Jabber, Skype, Twitter
# preferred vcard version: 3.0 / 4.0
preferred_version = 3.0
# Look into source vcf files to speed up search queries: yes / no
search_in_source_files = no
# skip unparsable vcard files: yes / no
skip_unparsable = no
Now
khard list shows all my contacts. So far so good. Apparently there
are some
awkward vCard compatibility
issues with creating or modifying
contacts from the
khard end. I ve tried adding one address from
~/.mutt/aliases using
khard and it seems to at least minimally work for
me, but I haven t explored this very much yet.
I had to install python3-vobject 0.9.4.1-1 from experimental to fix
eventable/vobject#39
saving certain vCard files.
Finally,
mutt integration. I already had
set query_command="lbdbq '%s'"
in
~/.muttrc, and I wanted to keep that in place since I still wanted to
use
LDAP querying as well. I had to write a very small amount of code for
this (perhaps I should contribute this to
lbdb upstream?), in
~/.lbdb/modules/m_khard:
#! /bin/sh
m_khard_query ()
khard email --parsable --remove-first-line --search-in-source-files "$1"
My full
~/.lbdb/rc now reads as follows (you probably won t want the
LDAP
stuff, but I ve included it here for completeness):
MODULES_PATH="$MODULES_PATH $HOME/.lbdb/modules"
METHODS='m_muttalias m_khard m_ldap'
LDAP_NICKS='debian canonical'
Next steps
I ve deleted one account from Google Contacts just to make sure that
everything still works (e.g. I can still search for it when composing a new
message), but I haven t yet deleted everything. I won t be adding anything
new there though.
I need to push everything from
~/.mutt/aliases into the new system. This
is only about 30 contacts so shouldn t take too long.
Overall this feels like a big improvement! It wasn t a trivial amount of
setup for just me, but it means I have both better usability for myself and
more independence from proprietary services, and I think I can add extra
users with much less effort if I need to.
Postscript
A day later and I ve consolidated all my accounts from Google Contacts and
~/.mutt/aliases into the new system, with the exception of one group that
I had defined as a
mutt alias and need to work out what to do with. This
all went smoothly.
I ve filed the new
lbdb module as
#866178, and the
python3-vobject bug as
#866181.
Today I noticed the internet buzz about a new alleged Linux malware called
OutlawCountry by the CIA, and leaked by Wikileaks.
The malware redirects traffic from the victim to a control server in order
to spy or whatever. To redirect this traffic, they use simple Netfilter NAT
rules injected in the kernel.
According to many sites commenting on the issue, is seems that there is
something wrong with the Linux kernel Netfilter subsystem, but I read the
leaked docs, and what they do is to load a custom kernel module in order to
be able to load Netfilter NAT table/rules with more priority than the default
ones (overriding any config the system may have).
Isn t that clear? The attacker is loading a custom kernel module as root
in your machine. They don t use Netfilter to break into your system. The
problem is not Netfilter, the problem is your whole machine being under
their control.
With root control of the machine, they could simply use any mechanism,
like kpatch or whatever, to replace your whole running kernel
with a new one, with full access to memory, networking, file system et al.
They probably use a rootkit or the like to take over the system.
The news media deliberately over-rates social media
News media outlets, including TV, radio and print, gain a significant benefit crowd-sourcing live information, free of charge, from the public on social media. It is only logical that they will cheer on social media sites and give them regular attention. Have you noticed that whenever Facebook's publicity department makes an announcement, the media are quick to publish it ahead of more significant stories about social or economic issues that impact our lives? Why do you think the media puts Facebook up on a podium like this, ahead of all other industries, if the media aren't getting something out of it too?
The tail doesn't wag the dog
One particular example is the news media's fascination with Donald Trump's Twitter account. Some people have gone as far as suggesting that this billionaire could have simply parked his jet and spent the whole of 2016 at one of his golf courses sending tweets and he would have won the presidency anyway. Suggesting that Trump's campaign revolved entirely around Twitter is like suggesting the tail wags the dog.
The reality is different: Trump has been a prominent public figure for decades, both in the business and entertainment world. During his presidential campaign, he 
I recently published a paper with 
A new version of the 
For each node, the prefix is known by its path from the root node and
the prefix length is the current depth.
A lookup in such a trie is quite simple: at each step, fetch the
nth bit of the 
Since some bits have been ignored, on a match, a final check is
executed to ensure all bits from the found entry are matching the
input 
The reduction on the average depth of the tree compensates the
necessity to handle those false positives. The insertion and deletion
of a routing entry is still easy enough.
Many routing systems are using Patricia trees:
Such a trie is called 
There are several structures involved:
The lookup time is loosely tied to the maximum depth. When the routing
table is densily populated, the maximum depth is low and the lookup
times are fast.
When forwarding at 10 Gbps, the time budget for a packet would be
about 50 ns. Since this is also the time needed for the route lookup
alone in some cases, we wouldn t be able to forward at line rate with
only one core. Nonetheless, the results are pretty good and they are
expected to scale linearly with the number of cores.
The measurements are done with a Linux kernel 4.11 from Debian
unstable. I have gathered performance metrics accross kernel versions
in 
The results are quite good. With only 256 MiB, about 2 million routes
can be stored!
For some reason, the relation is linear when the number of rules is
between 1 and 100 but the slope increases noticeably past this
threshold. The second graph highlights the negative impact of the
first rule (about 30 ns).
A common use of rules is to create virtual routers: interfaces
are segregated into domains and when a datagram enters through an
interface from domain A, it should use routing table A:
Yesterday I uploaded the first packages of 
I mentioned already in a 
The Debian Release Team in coordination with several other teams are preparing the last bits needed for releasing Debian 9 Stretch.
Please, be patient! Lots of steps are involved and some of them take some time, such as building the images, propagating the release through the mirror network, and rebuilding the Debian website so that "stable" points to Debian 9.
Follow the live coverage of the release on 
Apart from being somewhat slow, one of the downsides of the
original Raspberry Pi SoC was that it had an old ARM11 core which
implements the ARMv6 architecture. This was particularly
unfortunate as most common distributions (Debian, Ubuntu, Fedora,
etc) standardized on the ARMv7-A architecture as a minimum for
their ARM hardfloat ports. Which is one of the reasons for Raspbian
and the various other RPI specific distributions.
Happily, with the new 