I have been using
Replicant on the
Samsung SIII I9300 for over two years. I have written before on
taking a backup of the phone using rsync but recently I automated my setup as described below. This work was prompted by a screen accident with my phone that caused it to die, and I noticed that I hadn t taken regular backups. I did not lose any data this time, since typically all content I create on the device is immediately synchronized to my clouds. Photos are uploaded by the
ownCloud app,
SMS Backup+ saves SMS and call logs to my IMAP server, and I use
DAVDroid for synchronizing contacts, calendar and task lists with my instance of
ownCloud. Still, I strongly believe in regular backups of everything, so it was time to automate this.
For my use-case, taking backups of the phone whenever I connect it to one of my laptops is sufficient. I typically connect it to my laptops for charging at least every other day. My laptops are all running
Debian, but this should be applicable to most modern GNU/Linux system. This is not Replicant-specific, although you need a rooted phone. I thought that automating this would be simple, but I got to learn the ins and outs of
systemd and udev in the process and this ended up taking the better part of an evening.
I started out adding an udev rule and a small script, thinking I could invoke the backup process from the udev rule. However rsync would magically die after running a few seconds. After an embarrassing long debugging session, finally I
found someone with a similar problem which led me to a nice writeup on the topic
of running long-running services on udev events. I created a file
/etc/udev/rules.d/99-android-backup.rules with the following content:
ACTION=="add", SUBSYSTEMS=="usb", ENV ID_SERIAL_SHORT =="323048a5ae82918b", TAG+="systemd", ENV SYSTEMD_WANTS +="android-backup@$env ID_SERIAL_SHORT .service"
ACTION=="add", SUBSYSTEMS=="usb", ENV ID_SERIAL_SHORT =="4df9e09c25e75f63", TAG+="systemd", ENV SYSTEMD_WANTS +="android-backup@$env ID_SERIAL_SHORT .service"
The serial numbers correspond to the device serial numbers of the two devices I wish to backup. The
adb devices command will print them for you, and you need to replace my values with the values from your phones. Next I created a systemd service to describe a oneshot service. The file
/etc/systemd/system/android-backup@.service have the following content:
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/android-backup %I
The at-sign ( @ ) in the service filename signal that this is a service that takes a parameter. I m not enough of an udev/systemd person to explain these two files using the proper terminology, but at least you can pattern-match and follow the basic idea of them: the udev rule matches the devices that I m interested in (I don t want this to happen to all random Android devices I attach, hence matching against known serial numbers), and it causes a systemd service with a parameter to be started. The systemd service file describe the script to run, and passes on the parameter.
Now for the juicy part, the script. I have
/usr/local/sbin/android-backup with the following content.
#!/bin/bash
DIRBASE=/var/backups/android
export ANDROID_SERIAL="$1"
exec 2>&1 logger
if ! test -d "$DIRBASE-$ANDROID_SERIAL"; then
echo "could not find directory: $DIRBASE-$ANDROID_SERIAL"
exit 1
fi
set -x
adb wait-for-device
adb root
adb wait-for-device
adb shell printf "address 127.0.0.1\nuid = root\ngid = root\n[root]\n\tpath = /\n" \> /mnt/secure/rsyncd.conf
adb shell rsync --daemon --no-detach --config=/mnt/secure/rsyncd.conf &
adb forward tcp:6010 tcp:873
sleep 2
rsync -av --delete --exclude /dev --exclude /acct --exclude /sys --exclude /proc rsync://localhost:6010/root/ $DIRBASE-$ANDROID_SERIAL/
: rc $?
adb forward --remove tcp:6010
adb shell rm -f /mnt/secure/rsyncd.conf
This script warrant more detailed explanation. Backups are placed under, e.g.,
/var/backups/android-323048a5ae82918b/ for later off-site backup (you do backup your laptop, right?). You have to manually create this directory, as a safety catch to not wildly rsync data into non-existing directories. The script logs everything using syslog, so run a
tail -F /var/log/syslog& when setting this up. You may want to reduce verbosity of rsync if you prefer (replace
rsync -av with
rsync -a). The script runs
adb wait-for-device which you rightly guessed will wait for the device to settle. Next
adb root is invoked to get root on the device (reading all files from the system naturally requires root). It takes some time to switch, so another wait-for-device call is needed. Next a small rsyncd configuration file is created in
/mnt/secure/rsyncd.conf on the phone. The file tells rsync do listen on localhost, run as root, and use / as the path. By default, rsyncd is read-only so the host will not be able to upload any data over rsync, just read data out. Next rsync is started on the phone. The
adb forward command forwards port 6010 on the laptop to port 873 on the phone (873 is the default rsyncd port). Unfortunately, setting up the TCP forward appears to take some time, and
adb wait-for-device will not wait for that to complete, hence an ugly
sleep 2 at this point. Next is the rsync invocation itself, which just pulls in everything from the phone to the laptop, excluding some usual suspects. The somewhat cryptic
: rc $? merely logs the exit code of the rsync process into syslog. Finally we clean up the TCP forward and remove the rsyncd.conf file that was temporarily created.
This setup appears stable to me. I can plug in a phone and a backup will be taken. I can even plug in both my devices at the same time, and they will run at the same time. If I unplug a device, the script or rsync will error out and systemd cleans up.
If anyone has ideas on how to avoid the ugly temporary
rsyncd.conf file or the ugly
sleep 2, I m interested. It would also be nice to not have to do the adb root dance, and instead have the phone start the rsync daemon when connecting to my laptop somehow. TCP forwarding might be troublesome on a multi-user system, but my laptops aren t. Killing rsync on the phone is probably a good idea too. If you have ideas on how to fix any of this, other feedback, or questions, please let me know!
"Software Freedom Conservancy helps
promote, improve, develop, and defend Free, Libre, and Open Source
Software (FLOSS) projects. Conservancy provides a non-profit home and
infrastructure for FLOSS projects.", that is how Software Freedom
Conservancy defines itself. Organizations like Conservancy allow free
software developers to focus on what they do the best by doing
copyleft enforcement, taking care of legal aspects and provide many
services to its project
members.
Last August, Debian and Conservancy
announced a
partnership and formed the Copyright Aggregation Project where, among
other things, Conservancy will be able to hold copyrights for some
Debian works and ensure compliance with copyleft so that those works
remain in free software.
Recently, Conservancy launched a major fundraising campaign and needs
more individual supporters to gain more sustainable and independent
funding. This will allow the Conservancy to continue its efforts
towards convincing more companies to comply with free software
licenses such as the GPL and take legal actions when dialogue turns
out to be unsuccessful. Conservancy needs your support now, more than
ever!
Many Debian Developers and Contributors have already become
Conservancy
supporters. Please
consider signing up as a supporter on
https://sfconservancy.org/supporter/!
Following on 
.
What happened in the 
Here is my monthly update covering a large part of what I have been doing in the free software world (
Yesterday, 
Eric Raymond, author of 
This weekend I've mostly been tidying up some personal projects and things.
This post is intended to document some elements of workflow that I ve setup to manage videos produced for a MOOC, where different colleagues work collaboratively on a set of video sequences, in a remote way.
We are a team of several schools working on the same course, and we have an incremental process, so we need some collaboration over a quite long period of many remote authors, over a set of video sequences.
We re probably going to review some of the videos and make changes, so we need to monitor changes, and submit versions to colleagues on remote sites so they can criticize and get later edits. We may have more that one site doing video production. Thus we need to share videos along the flow of production, editing and revision of the course contents, in a way that is manageable by power users (we re all computer scientists, used to SVN or Git).
I ve decided to start an experiment with Git and 
The drawback is that we need manual intervention for starting the rsync, and also that we must make sure that the rsync target is ready to get a backup.
The target of the rsync on Linux is a git-annex clone using the default indirect mode, which handles the files as symlinks to the actual copies managed by git-annex inside the .git/ directory. But that ain t suitable to be compared to the origin of the rsync mirror which are plain files on the Windows computer.
We must then do a git-annex edit on the whole target of the rsync mirror before the rsync, so that the files are there as regular video files. This is costly, in terms of storage, and also copying time (our repo contains around 50 Gb, and the Linux host is a rather tiny laptop).
After the rsync, all the files need to be compared to the SHA256 known to git-annex so that only modified files are taken into account in the commit. We perform a git-annex add on the whole files (for new files having appeared at rsync time), and then a git-annex sync . That takes a lot of time, since all SHA256 computations are quite long for such a set of big files (the video rushes and edited videos are in HD).
So the process needs to be the following, on the target Linux host:
I would have preferred to have a working git-annex on windows, but that is a much more manageable process for me for now, and until we have more videos to handle in our repo that our Linux laptop can hold, we re quite safe.
Next steps will probably involve gardening the contents of the repo on the Linux host so we re only keeping copies of current files, and older copies are only kept on the 2 servers, in case of later need.
I hope this can be useful to others, and I d welcome suggestions on how to improve our process.
Get involved
After 
Dear readers,
There is a very useful tool for finding and merging shared permanent storage, and its name is fdupes. There was a terrible occurrence in the software after version 1.51, however. They removed the -L argument because too many people were complaining about lost data. It sounds like user error to me, and so I continue to use this one. I have to build from source, since the newer versions do not have the -L option.
