i3lock is a popular X11 screen lock utility. As far as
customization goes, it only allows one to set a background from a PNG
file. This limitation is part of the design of i3lock: its primary
goal is to keep the screen locked, something difficult enough with
X11. Each additional feature would increase the attack surface and
move away from this goal.1 Many are frustrated with these
limitations and extend i3lock through simple wrapper scripts or by
forking it.2 The first solution is usually safe, but the
second goes against the spirit of i3lock.
XSecureLock is a less-known alternative to i3lock. One of the
most attractive features of this locker is to delegate the screen
saver feature to another process. This process can be anything as long
it can attach to an existing window provided by XSecureLock, which
won t pass any input to it. It will also put a black window below it
to ensure the screen stays locked in case of a crash.
XSecureLock is shipped with a few screen savers, notably one using
mpv to display photos or videos, like the Apple TV aerial
videos. I have written my own saver using Python and GTK.3
It shows a background image, a clock, and the current
weather.4
Custom screen saver for XSecureLock
I add two patches over XSecureLock:
Sleep before mapping screen saver window. This patch prevents a
flash of black when starting XSecureLock by waiting a bit for the
screen saver to be ready before displaying it. As I am also using a
custom dimmer fading to the expected background before locking,
the flash of black was quite annoying for me. I have good hope this
patch will be accepted upstream.
Do not mess with DPMS/blanking. This patch prevents
XSecureLock from blanking the screen. I think this is solely the
role of the X11 DPMS extension. This makes the code simpler. I am
unsure if this patch would be accepted by upstream.
XSecureLock also delegates the authentication window to another
process, but I was less comfortable providing a custom one as it is a
bit more security-sensitive. While basic, the shipped authentication
application is fine by me.
I think people should avoid modifying i3lock code and use
XSecureLock instead. I hope this post will help a bit.
This Reddit post enumerates many of these alternatives.
Using GTK makes it a bit difficult to use some low-level
features, like embedding an application into an existing window.
However, the high-level features are easier, notably drawing an
image and a text with a shadow.
Weather is retrieved by another script running on a
timer and written to a file. The screen saver watches this file
for updates.
If your workstation is using full-disk encryption, you may want to
jump directly to your desktop environment after entering the
passphrase to decrypt the disk. Many display managers like GDM and
LightDM have an autologin feature. However, only GDM can run
Xorg with standard user privileges.
Here is an alternative using startx and a systemd service:
The unit starts after systemd-user-sessions.service, which enables
user logins after boot by removing the /run/nologin file.
With User=bernat, the unit is started with the identity of the
specified user. This implies that Xorg does not run with elevated
privileges.
With PAMName=login, the executed process is registered as a PAM
session for the login service, which includes pam_systemd.
This module registers the session to the systemd login manager.
To be effective, we also need to allocate a TTY with
TTYPath=/dev/tty8. When the TTY is active, the user is granted
additional access to local devices notably display, sound, keyboard,
mouse. These additional rights are needed to get Xorg working
rootless.1 The TERM environment variable is unset because
it would be set to linux by systemd as a result of attaching the
standard input to the TTY. Moreover, we inform pam_systemd we want
an X11 session with Environment=XDG_SESSION_TYPE=x11. Otherwise,
logind considers the session idle unless it receives input on
the TTY. Software relying on the idle hint from logind would be
ineffective.2
The UtmpIdentifier=tty8 and UtmpMode=user directives are just
a nice addition to register the session in /var/run/utmp.
The last step is to execute Xorg through startx. For logind to
allow Xorg to take control of the local devices, chvt 8 switches
to the allocated TTY.3StandardOutput=journal, combined
with the -verbose 3 -logfile /dev/null flags for Xorg, puts the
logs from the X server into the journal instead of using a file.
While equal to the default value, the Restart=no directive
highlights we do not want this unit to be restarted. This ensures
the loginless session is only available on boot. By default,
startx runs xinitrc. If you want to run Kodi instead, add
/usr/bin/kodi-standalone between startx and --.
Drop this unit in /etc/systemd/system/x11-autologin.service and
enable it with systemctl enable x11-autologin.service. Xorg is now
running rootless and logging into the journal. After using it for a
few months, I didn t notice any regression compared to LightDM with
autologin.
For more information on how logind provides access to
devices, see this blog post. The method names do not match the
current implementation, but the concepts are still correct. Xorg
takes control of the session when the TTY is active.
Xorg could change the type of the session itself after
taking control of it, but it does not.
There is some code in Xorg to do that, but it is
executed too late and fails with: xf86OpenConsole: VT_ACTIVATE
failed: Operation not permitted.
How to disable internal keyboard/touchpad when a cat arrives I m using an external keyboard (1) and mouse (2), but the laptop lid is usually still open for better cooling. That means the internal keyboard (3) and touchpad (4) made of comfortable materials are open to be used by a cat searching for warmth (7), in the obvious every time case that a normal non-heated nest (6) is not enough.
The problem is, everything goes chaotic at that point in the default configuration. The solution is to have quick shortcuts in my Dash to Dock (8) to both disable (10) and enable (9) keyboard and touchpad at a very rapid pace.It is to be noted that I m not disabling the touch screen (5) by default, because most of the time the cat is not leaning on it there is also the added benefit that if one forgets about the internal keyboard and touchpad disabling and detaches the laptop from the USB-C monitor (11), there s the possibility of using the touch screen and on-screen keyboard to type in the password and tap on the keyboard/touchpad enabling shortcut button again. If also touch screen was disabled, the only way would be to go back to an external keyboard or reboot.So here are the scripts. First, the disabling script (pardon my copy-paste use of certain string manipulation tools):
dconf write /org/gnome/desktop/peripherals/touchpad/send-events "'disabled'" sudo killall evtest sudo evtest --grab $(sudo libinput list-devices grep -A 1"AT Translated Set 2 keyboard" tail -n 1 sed 's/.*\/dev/\/dev/')& sudo evtest --grab $(sudo libinput list-devices grep -A 1"Dell WMI" tail -n 1 sed 's/.*\/dev/\/dev/')& sudo evtest --grab $(sudo libinput list-devices grep -A 1"Power" grep Kernel tail -n 1 sed 's/.*\/dev/\/dev/')& sudo evtest --grab $(sudo libinput list-devices grep -A 1"Power" grep Kernel head -n 1 sed 's/.*\/dev/\/dev/')& sudo evtest --grab $(sudo libinput list-devices grep -A 1"Sleep" grep Kernel tail -n 1 sed 's/.*\/dev/\/dev/')& sudo evtest --grab $(sudo libinput list-devices grep -A 1"HID" grep Kernel head -n 1 sed 's/.*\/dev/\/dev/')& sudo evtest --grab $(sudo libinput list-devices grep -A 1"HID" tail -n 1 sed 's/.*\/dev/\/dev/')& #sudo evtest --grab $(sudo libinput list-devices grep -A 1 "ELAN" tail -n 1 sed 's/.*\/dev/\/dev/') # Touch screen
And the associated ~/.local/share/applications/disable-internal-input.desktop:
With these, if I sense a cat or am just proactive enough, I press Super+9. If I m about to detach my laptop from the monitor, I press Super+8. If I forget the latter (usually this is the case) and haven t yet locked the screen, I just tap the enabling icon on the touch screen.
This post is part of a series called "PACKRAT". If this is the first post
you've found, it'd be worth reading the
intro post first and then looking over
all posts in the series.
In the last post, we left off at being able to send and recieve PACKRAT frames
to and from devices. Since we can transport IPv4 packets over the network,
let s go ahead and see if we can read/write
Ethernet frames from a Linux network
interface, and on the backend, read and write PACKRAT frames over the air.
This has the benifit of continuing to allow Linux userspace tools to work (like
cURL, as we ll try!), which means we don t have to do a lot of work to
implement higher level protocols or tactics to get a connection established
over the link.
Given that this post is less RF and more Linuxy, I m going to include more code
snippits than in prior posts, and those snippits are closer to runable Go, but
still not complete examples. There s also a lot of different ways to do this,
I ve just picked the easiest one for me to implement and debug given my
existing tooling for you, you may find another approach easier to implement!
Again, deviation here is very welcome, and since this segment is the least
RF centric post in the series, the pace and tone is going to feel different.
If you feel lost here, that s OK. This isn t the most important part of
the series, and is mostly here to give a concrete ending to the story arc.
Any way you want to finish your own journy is the best way for you to finish
it!
Implement Ethernet conversion code
This assumes an importable package with a Frame struct, which we can use to
convert a Frame to/from Ethernet. Given that the PACKRAT frame has a field that
Ethernet doesn t (namely, Callsign), that will need to be explicitly passed
in when turning an Ethernet frame into a PACKRAT Frame.
...// ToPackrat will create a packrat frame from an Ethernet frame.
funcToPackrat(callsign [8]byte, frame*ethernet.Frame) (*packrat.Frame, error)
varframeTypepackrat.FrameTypeswitchframe.EtherTypecaseethernet.EtherTypeIPv4:
frameType = packrat.FrameTypeIPv4default:
returnnil, fmt.Errorf("ethernet: unsupported ethernet type %x", frame.EtherType)
return&packrat.FrameDestination: frame.Destination,
Source: frame.Source,
Type: frameType,
Callsign: callsign,
Payload: frame.Payload,
, nil// FromPackrat will create an Ethernet frame from a Packrat frame.
funcFromPackrat(frame*packrat.Frame) (*ethernet.Frame, error)
varetherTypeethernet.EtherTypeswitchframe.Typecasepackrat.FrameTypeRaw:
returnnil, fmt.Errorf("ethernet: unsupported packrat type 'raw'")
casepackrat.FrameTypeIPv4:
etherType = ethernet.EtherTypeIPv4default:
returnnil, fmt.Errorf("ethernet: unknown packrat type %x", frame.Type)
// We lose the Callsign here, which is sad.
returnðernet.FrameDestination: frame.Destination,
Source: frame.Source,
EtherType: etherType,
Payload: frame.Payload,
, nil
Our helpers, ToPackrat and FromPackrat can now be used to transmorgify
PACKRAT into Ethernet, or Ethernet into PACKRAT. Let s put them into use!
Implement a TAP interface
On Linux, the networking stack can be exposed to userland using TUN or
TAP interfaces. TUN devices allow a
userspace program to read and write data at the Layer 3 / IP layer. TAP devices
allow a userspace program to read and write data at the Layer 2 Data Link /
Ethernet layer. Writing data at Layer 2 is what we want to do, since we re
looking to transform our Layer 2 into Ethernet s Layer 2 Frames. Our first job
here is to create the actual TAP interface, set the MAC address, and set the IP
range to our pre-coordinated IP range.
...import (
"net""github.com/mdlayher/ethernet""github.com/songgao/water""github.com/vishvananda/netlink"
)
...config:=water.ConfigDeviceType: water.TAPconfig.Name = "rat0"iface, err:=water.New(config)
...netIface, err:=netlink.LinkByName("rat0")
...// Pick a range here that works for you!
//
// For my local network, I'm using some IPs
// that AMPR (ampr.org) was nice enough to
// allocate to me for ham radio use. Thanks,
// AMPR!
//
// Let's just use 10.* here, though.
//
ip, cidr, err:=net.ParseCIDR("10.0.0.1/24")
...cidr.IP = iperr = netlink.AddrAdd(netIface, &netlink.AddrIPNet: cidr,
Peer: cidr,
)
...// Add all our neighbors to the ARP table
for_, neighbor:=rangeneighborsnetlink.NeighAdd(&netlink.NeighLinkIndex: netIface.Attrs().Index,
Type: netlink.FAMILY_V4,
State: netlink.NUD_PERMANENT,
IP: neighbor.IP,
HardwareAddr: neighbor.MAC,
)
// Pick a MAC that is globally unique here, this is
// just used as an example!
addr, err:=net.ParseMAC("FA:DE:DC:AB:LE:01")
...netlink.LinkSetHardwareAddr(netIface, addr)
...err = netlink.LinkSetUp(netIface)
varframe = ðernet.Framevarbuf = make([]byte, 1500)
forn, err:=iface.Read(buf)
...err = frame.UnmarshalBinary(buf[:n])
...// process frame here (to come)
...
Now that our network stack can resolve an IP to a MAC Address (via ip neigh
according to our pre-defined neighbors), and send that IP packet to our daemon,
it s now on us to send IPv4 data over the airwaves. Here, we re going to take
packets coming in from our TAP interface, and marshal the Ethernet frame into a
PACKRAT Frame and transmit it. As with the rest of the RF code, we ll leave
that up to the implementer, of course, using what was built during Part 2:
Transmitting BPSK symbols and Part 4: Framing
data.
...for// continued from above
n, err:=iface.Read(buf)
...err = frame.UnmarshalBinary(buf[:n])
...switchframe.EtherTypecase0x0800:
// ipv4 packet
pack, err:=ToPackrat(
// Add my callsign to all Frames, for now
[8]byte'K', '3', 'X', 'E', 'C' ,
frame,
)
...err = transmitPacket(pack)
......
Now that we have transmitting covered, let s go ahead and handle the recieve
path here. We re going to listen on frequency using the code built in
Part 3: Receiving BPSK symbols and
Part 4: Framing data. The Frames we decode from
the airwaves are expected to come back from the call
packratReader.Next in the code below, and the exact way that works
is up to the implementer.
...for// pull the next packrat frame from
// the symbol stream as we did in the
// last post
packet, err:=packratReader.Next()
...// check for CRC errors and drop invalid
// packets
err = packet.Check()
...ifbytes.Equal(packet.Source, addr)
// if we've heard ourself transmitting
// let's avoid looping back
continue// create an ethernet frame
frame, err:=FromPackrat(packet)
...buf, err:=frame.MarshalBinary()
...// and inject it into the tap
err = iface.Write(buf)
......
Phew. Right. Now we should be able to listen for PACKRAT frames on the
air and inject them into our TAP interface.
Putting it all Together
After all this work weeks of work! we can finally get around to putting
some real packets over the air. For me, this was an incredibly satisfying
milestone, and tied together months of learning!
I was able to start up a UDP server on a remote machine with an RTL-SDR dongle
attached to it, listening on the TAP interface s host IP with my defined MAC
address, and send UDP packets to that server via PACKRAT using my laptop,
/dev/udp and an Ettus B210, sending packets into the TAP interface.
Now that UDP was working, I was able to get TCP to work using two PlutoSDRs,
which allowed me to run the cURL command I pasted in the first post (both
simultaneously listen and transmit on behalf of my TAP interface).
It s my hope that someone out there will be inspired to implement their own
Layer 1 and Layer 2 as a learning exercise, and gets the same sense of
gratification that I did! If you re reading this, and at a point
where you ve been able to send IP traffic over your own Layer 1 / Layer 2,
please get in touch! I d be thrilled to hear all about it. I d love to link
to any posts or examples you publish here!
The first step when automating a network is to build the source of
truth. A source of truth is a repository of data that provides the
intended state: the list of devices, the IP addresses, the network
protocols settings, the time servers, etc. A popular choice is
NetBox. Its documentation highlights its usage as a source of
truth:
NetBox intends to represent the desired state of a network versus
its operational state. As such, automated import of live network
state is strongly discouraged. All data created in NetBox should
first be vetted by a human to ensure its integrity. NetBox can
then be used to populate monitoring and provisioning systems with a
high degree of confidence.
When introducing Jerikan, a common feedback we got was: you
should use NetBox for this. Indeed, Jerikan s source of truth is
a bunch of YAML files versioned with Git.
Why Git?
If we look at how things are done with servers and services, in a
datacenter or in the cloud, we are likely to find users of
Terraform, a tool turning declarative configuration files into
infrastructure. Declarative configuration management tools like
Salt, Puppet,1 or Ansible take care of server
configuration. NixOS is an alternative: it combines package
management and configuration management with a functional language to
build virtual machines and containers. When using a Kubernetes
cluster, people use Kustomize or Helm, two other declarative
configuration management tools. Tapped together, these tools implement
the infrastructure as code paradigm.
Infrastructure as code is an approach to infrastructure automation
based on practices from software development. It emphasizes
consistent, repeatable routines for provisioning and changing
systems and their configuration. You make changes to code, then use
automation to test and apply those changes to your systems.
Kief Morris, Infrastructure as Code, O Reilly.
A version control system is a central tool for infrastructure as code.
The usual candidate is Git with a source code management system like
GitLab or GitHub. You get:
Traceability and visibility
Git keeps a log of all changes: what, who, why, and when. With a
bit of discipline, each change is explained and self-contained. It
becomes part of the infrastructure documentation. When the support
team complains about a degraded experience for some customers over
the last two months or so, you quickly discover this may be related
to a change to an incoming policy in New York.
Rolling back
If a change is defective, it can be reverted quickly, safely, and
without much effort, even if other changes happened in the meantime.
The policy change at the origin of the problem spanned over three
routers. Reverting this specific change and deploying the
configuration let you solve the situation until you find a better
fix.
Branching, reviewing, merging
When working on a new feature or refactoring some part of the
infrastructure, a team member creates a branch and works on their
change without interfering with the work of other members. Once the
branch is ready, a pull request is created and the change is ready
to be reviewed by the other team members before merging. You
discover the issue was related to diverting traffic through an IX
where one ISP was connected without enough capacity. You propose and
discuss a fix that includes a change of the schema and the templates
used to declare policies to be able to handle this case.
Continuous integration
For each change, automated tests are triggered. They can detect
problems and give more details on the effect of a change. Branches
can be deployed to a test infrastructure where regression tests are
executed. The results can be synthesized as a comment in the pull
request to help the review. You check your proposed change does not
modify the other existing policies.
Why not NetBox?NetBox does not share these features. It is a database with a REST
and a GraphQL API. Traceability is limited: changes are not grouped
into a transaction and they are not documented. You cannot fork the
database. Usually, there is one staging database to test modifications
before applying them to the production database. It does not scale
well and reviews are difficult. Applying the same change to the
production database can be hazardous. Rolling back a change is
non-trivial.
Update (2021-11)
Nautobot, a fork of NetBox, will soon
address this point by using Dolt, an SQL database engine allowing
you to clone, branch, and merge, like a Git repository. Dolt is
compatible with MySQL clients. See Nautobots, Roll Back! for a
preview of this feature.
Moreover, NetBox is not usually the single source of truth. It
contains your hardware inventory, the IP addresses, and some topology
information. However, this is not the place you put authorized SSH
keys, syslog servers, or the BGP configuration. If you also use
Ansible, this information ends in its inventory. The source of
truth is therefore fragmented between several tools with different
workflows. Since NetBox 2.7, you can append additional data with
configuration contexts. This mitigates this point. The data is
arranged hierarchically but the hierarchy cannot be
customized.2Nautobot can manage configuration contexts in
a Git repository, while still allowing to use of the API to fetch
them. You get some additional perks, thanks to Git, but the
remaining data is still in a database with a different lifecycle.
Lastly, the schema used by NetBox may not fit your needs and you
cannot tweak it. For example, you may have a rule to compute the IPv6
address from the IPv4 address for dual-stack interfaces. Such a
relationship cannot be easily expressed and enforced in NetBox. When
changing the IPv4 address, you may forget the IPv6 address. The source
of truth should only contain the IPv4 address but you also want the
IPv6 address in NetBox because this is your IPAM and you need it to
update your DNS entries.
Why not Git?
There are some limitations when putting your source of truth in Git:
If you want to expose a web interface to allow an external team to
request a change, it is more difficult to do it with Git than
with a database. Out-of-the-box, NetBox provides a nice web
interface and a permission system. You can also write your own web
interface and interact with NetBox through its API.
YAML files are more difficult to query in different ways. For
example, looking for a free IP address is complex if they are
scattered in multiple places.
In my opinion, in most cases, you are better off putting the source of
truth in Git instead of NetBox. You get a lot of perks by doing
that and you can still use NetBox as a read-only view, usable by
other tools. We do that with an Ansible module. In the remaining
cases, Git could still fit the bill. Read-only access control can be
done through submodules. Pull requests can restrict write access: a
bot can check the changes only modify allowed files before
auto-merging. This still requires some Git knowledge, but many teams
are now comfortable using Git, thanks to its ubiquity.
Wikimedia manages its infrastructure with Puppet. They
publish everything on GitHub. Creative Commons uses
Salt. They also publish everything on GitHub.
Thanks to them for doing that! I wish I could provide more
real-life examples.
Being able to customize the hierarchy is key to avoiding
repetition in the data. For example, if switches are paired
together, some data should be attached to them as a group and not
duplicated on each of them. Tags can be used to partially work
around this issue but you lose the hierarchical aspect.
The libvirt and qemu versions in Debian Bullseye support a new feature that
allows for easier backup and recovery of virtual machines. Instead of using
snapshots for backup operation, its now possible to enable dirty
bitmaps. Other
hypervisors tend to call this changed block tracking .
Using the new backup
begin approach, its not
only possible to create live full backups (without having to create an
snapshot) but also track the changes between so called checkpoints,
which is very useful for incremental backups.
Over the course of the last few months, i have been working on a
simple backup and recovery utility called virtnbdbackup
It uses the pull based approach in the libvirt api set and currently
supports:
Thin provisioned full and incremental backups.
Compression (lz4).
Freezing the virtual machine guest fs via qemu-agent (if installed)
Multithreaded backup, if multiple disks are attached.
Point in time recovery to a given incremental backup.
Includes nbdkit plugin and utility that allows to map the thin provisioned
backup images to a block device, for single file or instant recovery (boot
the virtual machine directly from the backup image)
Preparing the virtual machines
The dirty bitmap feature is not enabled by default, users can enable
it by adding a new capability to a virtual machine configuration:
To finally enable the feature, power cycle virtual machine once.
Creating backups
By default, virtnbdbackup saves the virtual machine config, disks and
its logfiles to a given target directory. Its also possible to stream
the output into a uncompressed zip archive
Taking a backup is as simple as:
# virtnbdbackup -d vm2 -l full -o /tmp/WEEKLY_BACKUP/[..]
[2021-11-08 15:43:46] INFO virtnbdbackup - main [MainThread]: Backup jobs finished, stopping backup task.
[2021-11-08 15:43:46] INFO virtnbdbackup - main [MainThread]: Finished
# tree /tmp/WEEKLY_BACKUP/
/tmp/WEEKLY_BACKUP/
backup.full.11082021154343.log
checkpoints
virtnbdbackup.0.xml
sda.full.data
vm2.cpt
vmconfig.virtnbdbackup.0.xml
From that point on, its now possible to create incremental backups:
Restoring backups
The virtnbdrestore utility can be used to reconstruct the backup sets
into usable qcow images, like so:
# virtnbdrestore -a restore -i /tmp/WEEKLY_BACKUP/ -o /tmp/VM_RESTORE
Using the --until option its also possible to only reconstruct the
images to a certain checkpoint, allowing for point in time recovery.
Restoring single files
Via virtnbdmap you can map full backups back into an usable block device,
without having to reconstruct the complete backup image:
# virtnbdmap -f /tmp/WEEKLY_BACKUP/sda.full.data[..] INFO virtnbdmap - <module> [MainThread]: Done mapping backup image to [/dev/nbd0]
[..] INFO virtnbdmap - <module> [MainThread]: Press CTRL+C to disconnect
# fdisk -l /dev/nbd0
Disk /dev/nbd0: 2 GiB, 2147483648 bytes, 4194304 sectors
From here, you can either mount the disc and recover single files, or boot
from it via:
part = MIMEImage(open('/path/to/image', 'rb').read())
s = smtplib.SMTP('localhost') s.sendmail(from_addr, to_addr, msg.as_string()) s.quit()
will produce an email with empty body and the image as an attachment.The better way, ie to have the image as part of the body of the email, requires to write an HTML body that refers to that image:
import smtplib
from email.mime.multipart import MIMEMultipart from email.mime.text import MIMEText from email.mime.image import MIMEImage
# Define the image's ID as referenced in the HTML body above image.add_header('Content-ID', '<image1>') msg.attach(image)
s = smtplib.SMTP('localhost') s.sendmail(from_addr, to_addr, msg.as_string()) s.quit()
The trick is to define an image with a specific Content-ID and make that the only item in an HTML body: now you have an email with contains that specific image as the only content of the body, embedded in it.
Bonus point: if you want to take a snapshot of a webpage (which is kinda the reason i needed the code above) i found it extremely useful to use the Google PageSpeed Insights API; a good description on how to use that API with Python is available at this StackOverflow answer.
UPDATE (2020-12-26): I was made aware via email that some mail providers may not display images inline when the Content-ID value is too short (say, for example, Content-ID: 1). A solution that seems to work on most of the providers is using a sequence of random chars prefixed with a dot and suffixed with a valid mail domain.
part = MIMEImage(open('/path/to/image', 'rb').read())
s = smtplib.SMTP('localhost') s.sendmail(from_addr, to_addr, msg.as_string()) s.quit()
will produce an email with empty body and the image as an attachment.The better way, ie to have the image as part of the body of the email, requires to write an HTML body that refers to that image:
import smtplib
from email.mime.multipart import MIMEMultipart from email.mime.text import MIMEText from email.mime.image import MIMEImage
# Define the image's ID as referenced in the HTML body above image.add_header('Content-ID', '<image1>') msg.attach(image)
s = smtplib.SMTP('localhost') s.sendmail(from_addr, to_addr, msg.as_string()) s.quit()
The trick is to define an image with a specific Content-ID and make that the only item in an HTML body: now you have an email with contains that specific image as the only content of the body, embedded in it.
Bonus point: if you want to take a snapshot of a webpage (which is kinda the reason i needed the code above) i found it extremely useful to use the Google PageSpeed Insights API; a good description on how to use that API with Python is available at this StackOverflow answer.
UPDATE (2020-12-26): I was made aware via email that some mail providers may not display images inline when the Content-ID value is too short (say, for example, Content-ID: 1). A solution that seems to work on most of the providers is using a sequence of random chars prefixed with a dot and suffixed with a valid mail domain.
The Situation
I bought myself some USB microphones over ebay, I couldn t see any with USB type A connectors (the original USB connectors) and bought ones with USB-C connectors. I thought it would be good to have microphones that could work with recent mobile phones and with PCs, because surely it wouldn t be difficult to get an adaptor. I tested one of the microphones, it worked well on a phone.
I bought a pair of adaptors for USB A ports on a PC or laptop to USB-C (here s the link to where I bought them). I used one of the adaptors with a USB-C HDMI device which gave the following line from lsusb, I didn t try using a HDMI monitor on my laptop, having the device recognised was enough.
Bus 003 Device 002: ID 2109:0100 VIA Labs, Inc. USB 2.0 BILLBOARD
I tried connecting a USB-C microphone and Linux didn t recognise the existence of a USB device, I tried that on a PC and a laptop on multiple ports.
I wondered whether the description of the VIA BILLBOARD device as USB 2.0 was relevant to my problem. According to Big Mess O Wires USB-C has separate wires for USB 3.1 and USB 2 [1]. So someone could make a device that converts USB-A to USB-C with only USB-2 wires in place. I tested the USB-A to USB-C adaptor with the HDMI device in a USB SuperSpeed (IE 3.x) port and it still identified as USB 2.0. I suspect that the USB-C HDMI device is using all the high speed wires for DisplayPort data (with a conversion to HDMI) and therefore looks like a USB 2.0 device.
The Problem
I want to install a microphone in my workstation for long Zoom training sessions (7 hours in a day) that otherwise require me to use multiple Android devices as I don t have a device that will do 7 hours of Zoom without running out of battery. A new workstation with USB-C is unreasonably expensive. A PCIe USB-C card would give me the port at the back of the machine, I can t have the back of the machine near the microphone because it s too noisy.
If I could have a USB-C hub with reasonable length cables (the 1M cables typical for USB 2.0 hubs would be fine) connected to a USB-C port at the back of my workstation that would work. But there seems to be a great lack of USB-C hubs. NewBeDev has an informative post about the lack of USB-C hubs that have multiple USB-C ports [2]. There also seems to be a lack of USB-C hubs with cables longer than 20cm.
The Solution
I ended up ordering a Sades Wand gaming headset [3], that has over-ear headphones and an attached microphone which connects to the computer via USB 2.0. I gave the URL for the sades.com.au web site for reference but you will get a significantly better price by buying on ebay ($39+postage vs about $30 including postage).
I guess I won t be using my new USB-C microphones for a while.
This post was originally published in the Wikimedia Tech blog, authored by
Arturo Borrero Gonzalez.
NFS is a central piece of infrastructure that is essential to services like Toolforge.
Recently, the Cloud Services team at Wikimedia had been reviewing how we do NFS.
The current situation
NFS is a central piece of technology for some of the services that the
Wikimedia Cloud Services team offers to the community. We have several shares that power
different use cases: Toolforge user home directories live on NFS, and
Cloud VPS users can also access dumps using this protocol. The current setup involves
several physical hardware servers, with about 20TB of storage, offering shares over 10G links to
the cloud. For the system to be more fault-tolerant, we duplicate each share for redundancy using
DRBD.
Running NFS on dedicated hardware servers has traditionally offered us advantages: mostly on the
performance and the capacity fields.
As time has passed, we have been enumerating more and more reasons to review how we do NFS. For
one, the current setup is in violation of some of our internal rules regarding realm separation.
Additionally, we had been longing for additional flexibility managing our servers: we wanted to use
virtual machines managed by Openstack Nova. The DRBD-based high-availability system
required mostly a hand-crafted procedure for failover/failback. There s also some scalability
concerns as NFS is easy to grow up, but not to grow horizontally, and of course, we have to be able
to keep the tenancy setup while doing so, something that NFS does by using LDAP/Unix users and may
get complicated too when growing. In general, the servers have become too big to fail , clearly
technical debt, and it has taken us years to decide on taking on the task to rethink the
architecture. It s worth mentioning that in an ideal world, we wouldn t depend on NFS, but the
truth is that it will still be a central piece of infrastructure for years to come in services like
Toolforge.
Over a series of brainstorming meetings, the WMCS team evaluated the situation and sorted out the
many moving parts. The team managed to boil down the potential service future to two competing
options:
Adopt and introduce a new Openstack component into our cloud: Manila this was the right choice
if we were interested in a general NFS as a service offering for our Cloud VPS users.
Put the data on Cinder volumes and serve NFS from a couple of virtual machines created by hand
this was the right choice if we wanted something that required low effort to engineer and adopt.
Then we decided to research both options in parallel. For a number of reasons, the evaluation was
timeboxed to three weeks. Both ideas had a couple of points in common: the NFS data would be stored
on our Ceph farm via Cinder volumes, and we would rely on Ceph reliability to avoid using DRBD.
Another open topic was how to back up data from Ceph, to store our important bits in more than one
basket. We will get to the back up topic later.
The manila experiment
The Wikimedia Foundation was an early adopter of some Openstack components (Nova,
Glance, Designate, Horizon), but Manila was never
evaluated for usage until now. Our approach for this experiment was to closely follow the upstream
guidelines. We read the documentation and tried to understand the different setups you can build
with Manila. As we often feel with other Openstack components, the documentation doesn t perfectly
describe how to introduce a given component in your particular local setup. Here we use an
admin-controller flat-topology Neutron network. This network is shared by all tenants (or projects)
in our Openstack deployment. Also, Manila can use many different driver backends, for
things like NetApps or CephFS that we don t use , yet. After some research, the
generic driver was the one that seemed to better fit our use case. The generic driver
leverages Nova virtual machines instances plus Cinder volume to create and manage the shares. In
general, Manila supports two operational modes, whether it should create/destroy the share servers
(i.e, the virtual machine instances) or not. This option is called driver_handles_share_server (or
DHSS) and takes a boolean value.
We were interested in trying with DHSS=true, to really benefit from the potential of the setup.
NFS idea 6, original image in Wikitech
So, after sorting all these variables, we moved on with our initial testing. We built a PoC setup
as depicted in the diagram above, with the manila-share component running in a virtual machine
inside the cloud. The PoC led to us reporting several bugs upstream:
It s worth mentioning that the upstream community was extra-welcoming to us, and we re thankful for
that. However, at the end of our three-week period, our Manila setup still wasn t working as
expected. Your experience may change with other drivers perhaps the ZFSonLinux or the CephFS ones.
In general, we were having trouble making the setup work as expected, so we decided to abandon this
approach in favor of the other option we were considering at the beginning.
Simple virtual machine serving NFS
The alternative was to create a Nova virtual machine instance by hand and to configure it using
puppet. We have been investing in an automation framework lately, so the idea is to
not actually create the server by hand. Anyway, the data would be decoupled from the instance into
Cinder volumes, which led us to the question we left for later: How should we back up those
terabytes of important information? Just to be clear, the backup problem was independent of the above
options; with Manila we would still have had to solve the same challenge. We would like to see our
data be backed up somewhere else other than in Ceph. And that s exactly where we are at right now.
We ve been exploring different backup strategies and will finally use the Cinder backup
API.
Conclusion
The iteration will end with the dedicated NFS hardware servers being stopped, and the shares being
served from within the cloud. The migration will take some time to happen because we will check and
double-check that everything works as expected (including from the performance point of view)
before making definitive changes. We already have some plans to make sure our users experience as
little service impact as possible. The most troublesome shares will be those related to Toolforge.
At some point we will need to disallow writes to the NFS share, rsync the data out of the hardware
servers into the Cinder volumes, point the NFS clients to the new virtual machines, and then enable
writes again. The main Toolforge share has about 8TB of data, so this will take a while.
We will have more updates in the future. Who knows, perhaps our next-next iteration, in a couple of
years, will see us adopting Openstack Manila for good.
Featured image credit: File:(from break water) Manila Skyline panoramio.jpg, ewol, CC BY-SA 3.0This post was originally published in the Wikimedia Tech blog, authored
by Arturo Borrero Gonzalez.
Wasted quite some hours until I found a working Modeline in this
stack exchange post
so the ThinkPad works with a HDMI attached Samsung QHD display.
Internal display of the ThinkPad is a FHD display detected as eDP-1,
the external one is DP-3 and according to the packaging known by
Samsung as
S24A600NWU.
The auto deteced EDID modes for QHD - 2560x1440 - did not work at all, the display simply stays
dark. After a lot of back and forth with the i915 driver vs nouveau vs nvidia/nvidia-drm
with and without modesetting, the following Modeline did the magic:
Modelines for 50Hz and 60Hz generated with cvt 2560 1440 60 did not work, neither did the one
extracted with edid-decode -X from the hex blob found in .local/share/xorg/Xorg.0.log.
From the auto-detected Modelines FHD - 1920x1080 - did work. In case someone struggles
with a similar setup, that might be a starting point. Fun part, if I attach my several years old
Dell E7470 everything is just fine out of the box. But that one just has an Intel GPU and not
the unholy combination I've here:
Colson Whitehead's latest novel, Harlem Shuffle, was always going to be widely reviewed, if only because his last two books won Pulitzer prizes. Still, after enjoying both The Underground Railroad and The Nickel Boys, I was certainly going to read his next book, regardless of what the critics were saying indeed, it was actually quite agreeable to float above the manufactured energy of the book's launch.
Saying that, I was encouraged to listen to an interview with the author by Ezra Klein. Now I had heard Whitehead speak once before when he accepted the Orwell Prize in 2020, and once again he came across as a pretty down-to-earth guy. Or if I were to emulate the detached and cynical tone Whitehead embodied in The Nickel Boys, after winning so many literary prizes in the past few years, he has clearly rehearsed how to respond to the cliched questions authors must be asked in every interview. With the obligatory throat-clearing of 'so, how did you get into writing?', for instance, Whitehead replies with his part of the catechism that 'It seemed like being a writer could be a cool job. You could work from home and not talk to people.' The response is the right combination of cute and self-effacing... and with its slight tone-deafness towards enforced isolation, it was no doubt honed before Covid-19.
Harlem Shuffle tells three separate stories about Ray Carney, a furniture salesman and 'fence' for stolen goods in New York in the 1960s. Carney doesn't consider himself a genuine criminal though, and there's a certain logic to his relativistic morality. After all, everyone in New York City is on the take in some way, and if some 'lightly used items' in Carney's shop happened to have had 'previous owners', well, that's not quite his problem. 'Nothing solid in the city but the bedrock,' as one character dryly observes. Yet as Ezra pounces on in his NYT interview mentioned abov, the focus on the Harlem underworld means there are very few women in the book, and Whitehead's circular response ah well, it's a book about the criminals at that time! was a little unsatisfying. Not only did it feel uncharacteristically slippery of someone justly lauded for his unflinching power of observation (after all, it was the author who decided what to write about in the first place), it foreclosed on the opportunity to delve into why the heist and caper genres (from The Killing, The Feather Thief, Ocean's 11, etc.) have historically been a 'male' mode of storytelling.
Perhaps knowing this to be the case, the conversation quickly steered towards Ray Carney's wife, Elizabeth, the only woman in the book who could be said possesses some plausible interiority. The following off-hand remark from Whitehead caught my attention:
My wife is convinced that [Elizabeth] knows everything about Carney's criminal life, and is sort of giving him a pass. And I'm not sure if that's true. I have to have to figure out exactly what she knows and when she knows it and how she feels about it.
I was quite taken by this, although not simply due to its effect on the story it self. As in, it immediately conjured up a charming picture of Whitehead's domestic arrangements: not only does Whitehead's wife feel free to disagree with what one of Whitehead's 'own' characters knows or believes, but that Colson has no problem whatsoever sharing that disagreement with the public at large. (It feels somehow natural that Whitehead's wife believes her counterpart knows more than she lets on, whilst Whitehead himself imbues the protagonist's wife with a kind of neo-Victorian innocence.) I'm minded to agree with Whitehead's partner myself, if only due to the passages where Elizabeth is studiously ignoring Carney's otherwise unexplained freak-outs.
But all of these meta-thoughts simply underline just how emancipatory the Death of the Author can be. This product of academic literary criticism (the term was coined by Roland Barthes' 1967 essay of the same name) holds that the original author's intentions, ideas or biographical background carry no especial weight in determining how others should interpret their work. It is usually understood as meaning that a writer's own views are no more valid or 'correct' than the views held by someone else. (As an aside, I've found that most readers who encounter this concept for the first time have been reading books in this way since they were young. But the opposite is invariably true with cinephiles, who often have a bizarre obsession with researching or deciphering the 'true' interpretation of a film.) And with all that in mind, can you think of a more wry example of how freeing (and fun) nature of the Death of the Author than an author's own partner dissenting with their (Pulitzer Prize-winning) husband on the position of a lynchpin character?
The 1964 Harlem riot began after James Powell, a 15-year-old African American, was shot and killed by Thomas Gilligan, an NYPD police officer in front of 10s of witnesses. Gilligan was subsequently cleared by a grand jury.
As it turns out, the reviews for Harlem Shuffle have been almost universally positive, and after reading it in the two days after its release, I would certainly agree it is an above-average book. But it didn't quite take hold of me in the way that The Underground Railroad or The Nickel Boys did, especially the later chapters of The Nickel Boys that were set in contemporary New York and could thus make some (admittedly fairly explicit) connections from the 1960s to the present day that kind of connection is not there in Harlem Shuffle, or at least I did not pick up on it during my reading.
I can see why one might take exception to that, though. For instance, it is certainly true that the week-long Harlem Riot forms a significant part of the plot, and some events in particular are entirely contingent on the ramifications of this momentous event. But it's difficult to argue the riot's impact are truly integral to the story, so not only is this uprising against police brutality almost regarded as a background event, any contemporary allusion to the murder of George Floyd is subsequently watered down. It's nowhere near the historical rubbernecking of Forrest Gump (1994), of course, but that's not a battle you should ever be fighting.
Indeed, whilst a certain smoothness of affect is to be priced into the Whitehead reading experience, my initial overall reaction to Harlem Shuffle was fairly flat, despite all the action and intrigue on the page. The book perhaps belies its origins as a work conceived during quarantine after all, the book is essentially comprised of three loosely connected novellas, almost as if the unreality and mental turbulence of lockdown prevented the author from performing the psychological 'deep work' of producing a novel-length text with his usual depth of craft. A few other elements chimed with this being a 'lockdown novel' as well, particularly the book's preoccupation with the sheer physicality of the city compared to the usual complex interplay between its architecture and its inhabitants. This felt like it had been directly absorbed into the book from the author walking around his deserted city, and thus being able to take in details for the first time:
The doorways were entrances into different cities no, different entrances into one vast, secret city. Ever close, adjacent to all you know, just underneath. If you know where to look.
And I can't fail to mention that you can almost touch Whitehead's sublimated hunger to eat out again as well:
Stickups were chops they cook fast and hot, you re in and out. A stakeout was ribs fire down low, slow, taking your time.
[ ]
Sometimes when Carney jumped into the Hudson when he was a kid, some of that stuff got into his mouth. The Big Apple Diner served it up and called it coffee.
More seriously, however, the relatively thin personalities of minor characters then reminded me of the simulacrum of Zoom-based relationships, and the essentially unsatisfactory endings to the novellas felt reminiscent of lockdown pseudo-events that simply fizzle out without a bang. One of the stories ties up loose ends with: 'These things were usually enough to terminate a mob war, and they appeared to end the hostilities in this case as well.' They did? Well, okay, I guess.
The corner of 125th Street and Morningside Avenue in 2019, the purported location of Carney's fictional furniture store. Signage plays a prominent role in Harlem Shuffle, possibly due to the author's quarantine walks.
Still, it would be unfair to characterise myself as 'disappointed' with the novel, and none of this piece should be taken as really deep criticism. The book certainly was entertaining enough, and pretty funny in places as well:
Carney didn t have an etiquette book in front of him, but he was sure it was bad manners to sit on a man s safe.
[ ]
The manager of the laundromat was a scrawny man in a saggy undershirt painted with sweat stains. Launderer, heal thyself.
Yet I can't shake the feeling that every book you write is a book that you don't, and so we might need to hold out a little longer for Whitehead's 'George Floyd novel'. (Although it is for others to say how much of this sentiment is the expectations of a White Reader for The Black Author to ventriloquise the pain of 'their' community.)
Some room for personal critique is surely permitted. I dearly missed the junk food energy of the dry and acerbic observations that run through Whitehead's previous work. At one point he had a good line on the model tokenisation that lurks behind 'The First Negro to...' labels, but the callbacks to this idea ceased without any payoff. Similar things happened with the not-so-subtle critiques of the American Dream:
Entrepreneur? Pepper said the last part like manure. That s just a
hustler who pays taxes.
[ ]
One thing I ve learned in my job is that life is cheap, and when things start getting expensive, it gets cheaper still.
Ultimately, though, I think I just wanted more. I wanted a deeper exploration of how the real power in New York is not wielded by individual street hoodlums or even the cops but in the form of real estate, essentially serving as a synecdoche for Capital as a whole. (A recent take of this can be felt in Jed Rothstein's 2021 documentary, WeWork: Or the Making and Breaking of a $47 Billion Unicorn and it is perhaps pertinent to remember that the US President at the time this novel was written was affecting to be a real estate tycoon.). Indeed, just like the concluding scenes of J. J. Connolly's Layer Cake, although you can certainly pull off a cool heist against the Man, power ultimately resides in those who control the means of production... and a homespun furniture salesman on the corner of 125 & Morningside just ain't that. There are some nods to kind of analysis in the conclusion of the final story ('Their heist unwound as if it had never happened, and Van Wyck kept throwing up buildings.'), but, again, I would have simply liked more.
And when I attempted then file this book away into the broader media landscape, given the current cultural visibility of 1960s pop culture (e.g. One Night in Miami (2020), Judas and the Black Messiah (2021), Summer of Soul (2021), etc.), Harlem Shuffle also seemed like a missed opportunity to critically analyse our (highly-qualified) longing for the civil rights era. I can certainly understand why we might look fondly on the cultural products from a period when politics was less alienated, when society was less atomised, and when it was still possible to imagine meaningful change, but in this dimension at least, Harlem Shuffle seems to merely contribute to this nostalgic escapism.
This Perl module converts POD documents to thread, the macro language I
use for my static site builder (about which, more in another software
release coming up shortly). This release fixes a whole ton of
long-standing problems.
Remove support for parsing CVS $Id$ strings and emitting
\id commands.
Stop wrapping output text.
Fix internal section links.
Fix conversion of nested =over blocks.
Fix conversion of =over blocks without an =item.
Fix links to headings containing markup.
Fix metacharacter escaping in headings, contents, and navbar.
Fix handling of =for thread and =begin thread.
Fix non-breaking spaces in navbar section names.
Fix internal links whose anchors are wrapped.
Output the document before erroring out if POD errors were seen.
Suppress \signature if the input document was empty.
Add bugtracker, homepage, and repository package metadata.
In brief, a lot of the POD implementation was previously done by chasing
bugs rather than testing comprehensively, as reflected by the 65% code
coverage in the previous release. The test suite now achieves about 95%
code coverage (most of the rest is obscure error handling around
encoding) and cleans up a bunch of long-standing problems with internal
links.
I had previously punted entirely on section links containing markup, and
as a result the section links shown in the navigation bar or table of
contents were missing the markup and headings containing thread
metacharacters were mangled beyond recognition. That was because I was
trying to handle resolving links using regexes (after I got rid of the
original two-pass approach that required a driver script). This release
uses Text::Balanced instead, the same parsing module used by my static
site generator, to solve the problem (mostly) correctly. (I think there
may still be a few very obscure edge cases, but I probably won't encounter
them.)
The net result should hopefully be better conversion of my software
documentation, including INN documentation, when I regenerate my site
(which will be coming soon).
You can get the latest release from CPAN or from the
Pod::Thread distribution page.
It seems that every cloud service of note has a free tier nowadays and the Oracle Cloud is the latest that I ve discovered (thanks to r/homelab which I highly recommend reading). Here s Oracle s summary of what they offer for free [1].
Oracle s always free tier (where presumable always is defined as until we change our contract ) currently offers ARM64 VMs to a total capacity of 4 CPU cores, 24G of RAM, and 200G of storage with a default VM size of 1/4 that (1 CPU core and 6G of RAM). It also includes 2 AMD64 VMs that each have 1G of RAM, but a 64bit VM with 1G of RAM isn t that useful nowadays.
Web Interface
The first thing to note is that the management interface is a massive pain to use. When a login times out for security reasons it redirects to a web page that gives a 404 error, maybe the redirection works OK if you are using it when it times out, but if you go off and spend an hour doing something else you will return to a 404 page. A web interface should never refer you to a page with a 404.
There doesn t seem to be a way of bookmarking the commonly used links (as AWS does) and the set of links on the left depend on the section you are in with no obvious way of going between sections. Sometimes I got stuck in a set of pages about authentication controls (the identity cloud ) and there seems to be no link I could click on to get me back to cloud computing, I had to go to a bookmarked link for the main cloud login page. A web interface should never force the user to type in the main URL or go to a bookmark, you should be able to navigate from every page to every other page in a logical manner. An advanced user might have their own bookmarks in their browser to suit their workflow. But a beginner should be able to go to anywhere without breaking the session.
Some parts of the interface appear to be copied from AWS, but unfortunately not the good parts. The way AWS manages IP access control is not easy to manage and it s not clear why packets are dropped, Oracle copies all this. On the upside Oracle has some good Datadog style analytics so for a new deployment you can debug IP access control by seeing records of rejected packets. Just to make it extra annoying when you create a rule with multiple ports specified the web interface will expand it out to multiple rules for one port each, having ports 80 and 443 on separate lines doesn t make things easier. Also it forces you to have IPv4 and IPv6 as separate rules, so if you want HTTP and HTTPS on both IPv4 and IPv6 (a common requirement) then you need 4 separate rules.
One final annoying thing is that the web interface doesn t make your previous settings a default. As I ve created many ARM images and haven t created a single AMD image it should know that the probability that I want to create an AMD image is very low and stop defaulting to that.
Recovery
When trying a new system you will inevitably break things and have to recover things. The way to recover from a configuration error that prevents your VM from booting and getting to a state of allowing a login is to go to stop the VM, then go to the Boot volume section under Resources and use the settings button to detach the boot volume. Then you go to another VM (which must be running), go to the Attached block volumes menu and attach it as Paravirtualised (not iSCSI and not default which will probably be iSCSI). After some time the block device will appear and you can mount it and do stuff to it. Then after umounting it you detach it from the recovery VM and attach it again to the original VM (where it will still have an entry in the Boot volume section) and boot the original VM.
As an aside it s really annoying that you can t attach a volume to a VM that isn t running.
My first attempt at image recovery started with making a snapshot of the Boot volume, this didn t work well because the image uses EFI and therefore GPT and because the snapshot was larger than the original block device (which incidentally was the default size). I admit that I might have made a mistake making the snapshot, but if so it shouldn t be so easy to do. With GPT if you have a larger block device then partitioning tools complain about the backup partition table not being found, and they complain even more if you try to go back to the smaller size later on. Generally GPT partition tables are a bad idea for VMs, when I run the host I don t use partition tables, I have a separate block device for each filesystem or swap space.
Snapshots aren t needed for recovery, they don t seem to work very well, and if it s possible to attach a snapshot to a VM in place of it s original Boot volume I haven t figured out how to do it.
Console Connection
If you boot Oracle Linux a derivative of RHEL that has SE Linux enabled in enforcing mode (yay) then you can go to the Console connection . The console is a Javascript console which allows you to login on a virtual serial console on device /dev/ttyAMA0. It tells you to type help but that isn t accepted, you have a straight Linux console login prompt.
If you boot Ubuntu then you don t get a working serial console, it tells you to type help for help but doesn t respond to that.
It seems that the Oracle Linux kernel 5.4.17-2102.204.4.4.el7uek.aarch64 is compiled with support for /dev/ttyAMA0 (the default ARM serial device) while the kernel 5.11.0-1016-oracle compiled by Oracle for their Ubuntu VMs doesn t have it.
Performance
I haven t done any detailed tests of VM performance. As a quick test I used zstd to compress a 154MB file, on my home workstation (E5-2620 v4 @ 2.10GHz) it took 11.3 seconds of CPU time to compress with zstd -9 and 7.2s to decompress. On the Oracle cloud it took 7.2s and 5.4s. So it seems that for some single core operations the ARM CPU used by the Oracle cloud is about 30% to 50% faster than a E5-2620 v4 (a slightly out of date server processor that uses DDR4 RAM).
If you ran all the free resources in a single VM that would make a respectable build server. If you want to contribute to free software development and only have a laptop with 4G of RAM then an ARM build/test server with 24G of RAM and 4 cores would be very useful.
Ubuntu Configuration
The advantage of using EFI is that you can manage the kernel from within the VM. The default Oracle kernel for Ubuntu has a lot of modules included and is compiled with a lot of security options including SE Linux.
Competitors
https://aws.amazon.com/free
AWS offers 750 hours (just over 31 days) per month of free usage of a t2.micro or t3.micro EC2 instance (which means 1GB of RAM). But that only lasts for 12 months and it s still only 1GB of RAM. AWS has some other things that could be useful like 1 million free Lambda requests per month. If you want to run your personal web site on Lambda you shouldn t hit that limit. They also apparently have some good offers for students.
https://cloud.google.com/free
The Google Cloud Project (GCP) offers $300 of credit.
https://cloud.google.com/free/docs/gcp-free-tier#free-tier-usage-limits
GCP also has ongoing free tier usage for some services. Some of them are pretty much unlimited use (50GB of storage for Cloud Source Repositories is a heap of source code). But for VMs you get the equivalent of 1*e2-micro instance running 24*7. A e2-micro has 1G of RAM. You also only get 30G of storage and 1GB of outbound data. It s clearly not as generous an offer as Oracle, but Oracle is the underdog so they have to try harder.
https://azure.microsoft.com/en-us/free/
Azure appears to be much the same as AWS, free Linux VM for a year and then other less popular services free forever (or until they change the contract).
https://www.ibm.com/cloud/free
The IBM cloud free tier is the least generous offer, a VM is only free for 30 days. But what they offer for 30 days is pretty decent. If you want to try the IBM cloud and see if it can do what your company needs then this will do well. If you want to have free hosting for your hobby stuff then it s no good.
Oracle seems like the most generous offer if you want to do stuff, but also one of the least valuable if you want to learn things that will help you at a job interview. For job interviews AWS seems the most useful and then GCP and Azure vying for second place.
A quick note to self to remind how I do backups of my Android device with rsync (and adb).
I have followed this guide: How to use rsync over USB on Android with adb
My personal notes:
I have Lineage so I have rsync in my Android device already installed
I run Debian stable (buster, for now) on my laptop, with adb installed
My /sdcard/rsyncd.conf file:
address = 127.0.0.1 port = 1873 uid = 0 gid = 0 [root] path = / use chroot = false read only = false'
The command:
adb shell /data/local/tmp/rsync --daemon --no-detach --config=/sdcard/rsyncd.conf --log-file=/proc/self/fd/2
didn't work, produced this message: "@ERROR: protocol startup error" so I ended up doing:
adb shell rsync --daemon --no-detach --config=/sdcard/rsyncd.conf --log-file=/sdcard/rsync.log
and opened another tab to perform the rsync commands from my laptop:
rsync -av --progress --stats rsync://localhost:6010/root/storage . rsync -av --progress --stats rsync://localhost:6010/root/data .
Then I saw that rsync was copying the symlinks instead of their contents: /storage/self/primary was a broken link to /mnt/user/0/primary
So I ran again the commands with -LK:
rsync -av --progress --stats -LK rsync://localhost:6010/root/storage . rsync -av --progress --stats -LK rsync://localhost:6010/root/data .
and now I have a copy of all the files I'm interested. In addition to this, I run an adb backup of the system:
adb backup -f ./adb_backup_apk_shared_all_system.ad -apk -shared -all -system
and I think that's all that I need for the case I want to remove stuff from my phone or some disaster happens.
Tonight I m provisioning a new virtual machine at
Hetzner and I wanted to share how
Consfigurator is helping
with that. Hetzner have a Debian buster image you can start with, as you d
expect, but it comes with things like cloud-init, preconfiguration to use
Hetzner s apt mirror which doesn t serve source packages(!), and perhaps other
things I haven t discovered. It s a fine place to begin, but I want all the
configuration for this server to be explicit in my Consfigurator consfig, so
it is good to start with pristine upstream Debian. I could boot one of
Hetzner s installation ISOs but that s slow and manual. Consfigurator can
replace the OS in the VM s root filesystem and reboot for me, and we re ready
to go.
Here s the configuration:
(defhost foo.silentflame.com (:deploy ((:ssh :user "root") :sbcl))
(os:debian-stable "buster" :amd64)
;; Hetzner's Debian 10 image comes with a three-partition layout and boots
;; with traditional BIOS.
(disk:has-volumes
(physical-disk
:device-file "/dev/sda" :boots-with '(grub:grub :target "i386-pc")))
(on-change (installer:cleanly-installed-once
nil
;; This is a specification of the OS Hetzner's image has, so
;; Consfigurator knows how to install SBCL and debootstrap(8).
;; In this case it's the same Debian release as the replacement.
'(os:debian-stable "buster" :amd64))
;; Clear out the old OS's EFI system partition contents, in case we can
;; switch to booting with EFI at some point (if we wanted we could specify
;; an additional x86_64-efi target above, and grub-install would get run
;; to repopulate /boot/efi, but I don't think Hetzner can boot from it yet).
(file:directory-does-not-exist "/boot/efi/EFI")
(apt:installed "linux-image-amd64")
(installer:bootloaders-installed)
(fstab:entries-for-volumes
(disk:volumes
(mounted-ext4-filesystem :mount-point "/")
(partition
(mounted-fat32-filesystem
:mount-options '("umask=0077") :mount-point "/boot/efi"))))
(file:lacks-lines "/etc/fstab" "# UNCONFIGURED FSTAB FOR BASE SYSTEM")
(file:is-copy-of "/etc/resolv.conf" "/old-os/etc/resolv.conf")
(mount:unmounted-below-and-removed "/old-os"))
(apt:mirror "http://ftp.de.debian.org/debian")
(apt:no-pdiffs)
(apt:standard-sources.list)
(sshd:installed)
(as "root" (ssh:authorized-keys +spwsshkey+))
(sshd:no-passwords)
(timezone:configured "Etc/UTC")
(swap:has-swap-file "2G")
(network:clean-/etc/network/interfaces)
(network:static "enp1s0" "xxx.xxx.xxx.xxx" "xxx.xxx.1.1" "255.255.255.255"))
Here the :HOP parameter specifies the IP address of the new machine, as
DNS hasn t been updated yet. Consfigurator installs SBCL and debootstrap(8),
prepares a minimal system, replaces the contents of /, gets to work
applying the other properties, and then reboots. This gets us a properly
populated fstab:
(slightly doctored for more readable alignment)
There s ordering logic so that the swapfile will end up after whatever
filesystem contains it; a UUID is used for ext4 filesystems, but for fat32
filesystems, to be safe, a PARTUUID is used.
The application of (INSTALLER:BOOTLOADERS-INSTALLED) handles calling both
update-grub(8) and grub-install(8), relying on the metadata specified
about /dev/sda. Next time we execute Consfigurator against the machine,
it ll ignore all the property applications attached to the application of
(INSTALLER:CLEANLY-INSTALLED-ONCE) with ON-CHANGE, and just apply
everything following that block.
There are a few things I don t have good solutions for. When you boot
Hetzner s image the primary network interface is eth0, but then for a
freshly debootstrapped Debian you get enp1s0, and I haven t got a good way
of knowing what it ll be (if you know it ll have the same name, you can use
(NETWORK:PRESERVE-STATIC-ONCE) to create a file in
/etc/network/interfaces.d based on the current default route and corresponding
interface).
Another tricky thing is SSH host keys. It s easy to use Consfigurator to add
host keys to your laptop s ~/.ssh/known_hosts, but in this case the host
key changes back and forth from whatever the Hetzner image has and the newly
generated key you get afterwards. One option might be to copy the old host
keys out of /old-os before it gets deleted, like how /etc/resolv.conf
is copied.
This work is based on Propellor s equivalent
functionality.
I think my approach to handling /etc/fstab and bootloader installation is an
improvement on what Joey does.
This blog post is about my observations of some of the plants in my home garden.
While still a n00b on the subject, these notes are my observations and experiences over days, weeks and months. Thankfully, with the capability to take frequent pictures, it has been easy to do an assessment and generate a report of some of these amazing behaviors of plants, in an easy timeline order; all thanks to the EXIF data embedded.
This has very helpfully allowed me to record my, otherwise minor observations, into great detail; and make some sense out of it by correlating the data over time.
It is an emotional experience. You see, plants are amazing. When I sow a sapling, water it, feed it, watch it grow, prune it, medicate it, and what not; I build up affection towards it.
Though, at the same time, to me it is a strict relationship, not too attached; as in it doesn t hurt to uproot a plant if there is a good reason. But still, I find some sort of association to it.
With plants around, it feels I have a lot of lives around me. All prospering, communicating, sharing. And communicate they do. What is needed is just the right language to observe and absorb their signals and decipher what they are trying to say.
Devastation
How in this world, when you are caring for your plants, can it transform:
From This
Healthy Mulberry Plant
Healthy Mulberry Plant
Healthy Bael Plant
To This
Dead Mulberry Plant
Very Sick Bael Plant
With emotions involved, this can be an unpleasant experience.
Bael is a dear plant to me. The plant as a whole has religious values (Shiva). As well, its fruits have lots of health benefits, especially for the intestines. Its leaves have a lot of medicinal properties.
When I planted the Bael, there were a lot of emotions that went along.
On the other hand, the Mulberry is something I put in with a lot of enthusiasm. Mulberries are now rare to find, especially in urban locations. For one, they have a very short shelf life; But more than that, the way lifestyles are heading towards, I was always worried if my children would ever have a day to see and taste these fruits.
The mulberry that I planted, yielded twice; once very soon when I had planted and second, before it died. Infact, it died while during its second yield phase.
It was quite saddening to see that happen. It made me wonder why it happened. I had been caring for the plants fairly well. Watering them timely, feeding them the right amount of nutrients. They were getting a good amount of sun.
But still their health was deteriorating. And then the demise of the Mulberry. Many thoughts hit my mind.
I consulted the claimed experts in the domain, the maali, the gardener. I got a very vague answer; there must be termites in the soil. It didn t make much sense to me. I mean if there are termites they d hit day one. They won t sleep for months and just wake up one fine day and start attacking the roots of particular plants; not all.
I wasn t convinced with the termite theory; But still, give the expert the blind hand, I went with his word.
When my mulberry was dead, I dug its roots. Looking for proof, to see if there were any termites, I uprooted it. But I couldn t find any trace of termites. And the plant next to it was perfectly healthy and blossoming. So I was convinced that it wasn t the termite but something else.
But else what ? I still didn t have an answer to that.
Thinking
The Corona pandemic had embraced and there was a lot to worry, and worry not any, if you change the perspective. With plants around in my home, and our close engagement with them, and the helplessness that I felt after seeking help from the experts, it was time again; to build up some knowledge on the subject.
But how ? How do you go about a subject you have not much clue about ? A subject which has always been around in the surrounding but very seldom have I dedicated focused thought to it. To be honest, the initial thought of diving on the subject made me clueless. I had no idea where to begin with. But, so, as has been my past history, I chose to take it as a curiosity.
I gathered some books, skimmed through a couple of pages. Majority of the books I got hold of were about DIYs and How to do Home Gardening types. It was a decent introduction to a novice but my topic of curiosity was different.
Thankfully, with the Internet, and YouTube in particular, a lot of good stuff is available as documentary videos. While going through some, I came across a video which mentioned about carnivore plants. Like, for example, this one.
Carnivore Plant
This got me thinking that there could be a possibility of something similar, that did the fate to my Mulberry plant. But who did it ? And how to dive further on this suspicion ? And most of all, if that thought of possibility was actually the reality. Or was I just hitting in the dark ?
Beginning
To put some perspective, here s how it started. When we moved into our home, the gardener put in a couple of plants stock, as part of the property handover. Now, I don t exactly recollect the name of the plants that came in stock, neither English nor Hindi; But at my neighbor s place, the plant is still there.
Here are some of the pictures of this beauty. But don t just go by the looks as looks can be deceiving
Dominating Plant
Dominating Plant
Dominating Plant
We hadn t put any serious thought about the plants we were offered by the gardener. After all, we never had ever thought of any mishap either.
Plants we planted
Apart from what was offered by the builder/gardener as part of the property handover, in over the next 6 months of we moving in, I planted 3 tree type plants.
Mulberry
Bael
Rudraksha
The Mulberry, as I have described so far, died a tragic death.
Bael, on the other hand, fought hard. But very little did we know that the plant was struggling the fight. Our impression was that we must have been given a bad breed of the plant. Or maybe the termite theory had some truth.
For the Rudraksha plant, the growth was slow. This was the very first time I had seen a Rudraksha plant, so I had no clue of what its growth rate could be, and what to expect out of it. I wasn t sure if the local climate suited the plant. A quick search showed no objections to the plant in the local climate, but that was it. So my theory has been to put in the plant, and observe.
Here s what my Rudraksha plant looked like during the initial days/weeks of its settlement
Rudraksha Plant
The Hint
Days passed and so on. Not much had progressed in gathering information. The plant s health was usual; deteriorating at a slow pace.
On day, thinking of the documentaries I had been watching, it hit my mind about the plant behavior.
Plants can be Carnivores.
Plants can be Aggressive.
Plants can be Invaders.
Plants can be Territorial.
There are many plants where their aggression can be witnessed with bare human eyes. Like creepers. Some of them are good at spreading tentacles, grabbing onto other plants' stems and branches and spread above it. This was my hint from the documentaries. That s one of the many ways plants set their dominance. That is what hit my mind that if plants are aggressive on the out, underneath the soil, they should be having similar behavior.
I mean, what we see as humans is just a part of the actual plant. More than half of the actual plant is usually underneath the soil, in most plants.
So there s a high chance to get more information out, if you dig the soil and look the roots.
The Digging
As I mentioned earlier, I do establish bindings, emotions and attachments. But not much usually comes in the way to curiosity.
To dig further on the theory that the problem was elsewhere, with within the plants ecosystem, we needed to pick on another subject - the plant.
And the plant we chose was the plant which was planted in the initial offering to us, when we moved into our home. It was the same plant breed which was neighboring all our newly planted trees: Rudraksha, Mulberry and Bael.
If you look closely into the pictures above of these plants, you ll notice the stem of another plant, the Territorial Dominator, is close-by to these 3 plants. That s because the gardener put in a good number of them to get his action item complete.
So we chose to dig and uproot one of those plant to start with. Now, while they may look gentle on the outside, with nice red colored tiny flowers, these plants were giants underneath. Their roots were huge. It took some sweat shredding to single-handedly remove them.
Dominating Plant Uprooted
Dominating Plant Uprooted
Dominating Plant Uprooted
Dominating Plant Uprooted
Today is brighter
Bael
I ll let the pictures do the initial talking today.
Healthy Bael
Healthy Bael
Healthy Bael
Healthy Bael
Healthy Bael
The above ones are pictures of the same Bael plant, which had struggled to live, for almost 14 months. Back then, this plant was starved of its resources. It was dying a slow death out of starvation. After we uprooted the other dominant species, the Bael has recovered and has regained its charm.
In the pictures above of the Bael plant, you can clearly mark out the difference in its stem. The dark colored one is from its months of struggle, while the bright green is from now where it is well nourished and regained its health.
Mulberry
As for the Mulberry, I couldn t save it. But I later managed to get another one. But it turns out I didn t take good, full length pictures of the new mulberry when I planted. The only picture I have is this:
Second Mulberry Plant
I recollect when I brought it home, it was around 1 - 1.5 feet in length.
This is where I have it today: Majestically standing, 12 feet and counting
Second Mulberry Plant 7 feet tall
Rudraksha
Then:
Rudraksha Plant
And now:
I feel quite happy about
Rudraksha Plant
Rudraksha Plant
Rudraksha Plant
All these plants are on the very same soil with the very same care taker. What has changed is my experience and learning.
Plant Co-Existence
Plant co-existence is a difficult topic. My knowledge on plants is very limited in general, and co-existence is something tricky, unexplored, at times invisible (when underneath the soil). So it is a difficult topic. So far, what I ve learnt is purely observations, experiences and hints from the documentaries.
There surely are many many plants that co-exist very well. A good example is my Bael plant itself, which is healthily co-sharing its space with 2 other Croton plants. Same goes for the Rudraksha, which has a close-by neighbor in an Adenium and an Allamanda.
The plant world is mesmerizing. How they behave, communicate and many many more signs. There s so much to observe, learn, explore and document. I hope to have more such observations and experiences to share
"Again, we see there is nothing you can possess which I cannot take away."
The cinema was a rare and expensive treat in my youth, so I first came across Raiders of the Lost Ark by recording it from television onto a poor quality VHS. I only mention this as it meant I watched a slightly different film to the one intended, as my copy somehow missed off the first 10 minutes. For those not as intimately familiar with the film as me, this is just in time to see a Belloq demand Dr. Jones hand over the Peruvian head (see above), just in time to learn that Indy loathes snakes, and just in time to see the inadvertent reproduction of two Europeans squabbling over the spoils of a foreign land.
What this truncation did to my interpretation of the film (released thirty years ago today on June 19th 1981) is interesting to explore. Without Jones' physical and moral traits being demonstrated on-screen (as well as missing the weighing the gold head and the rollercoaster boulder scene), it actually made the idea of 'Indiana Jones' even more of a mythical archetype. The film wisely withholds Jones' backstory, but my directors cut deprived him of even more, and counterintuitively imbued him with even more of a legendary hue as the elision made his qualities an assumption beyond question. Indiana Jones, if you can excuse the clich , needed no introduction at all.
Good artists copy, great artists steal. And oh boy, does Raiders steal. I've watched this film about twenty times over the past two decades and it's now firmly entered into my personal canon. But watching it on its thirtieth anniversary was different not least because I could situate it in a broader cinematic context. For example, I now see the Gestapo officer in Major Strasser from Casablanca (1942), in fact just as I can with many of Raiders' other orientalist tendencies: not only in its breezy depictions of backwards sand people, but also of North Africa as an entrep t and playground for a certain kind of Western gangster. The opening as well, set in an equally reductionist pseudo-Peru, now feels like Werner Herzog's Aguirre, the Wrath of God (1972) but without, of course, any self-conscious colonial critique.
The imagery of the ark appears to be borrowed from James Tissot's The Ark Passes Over the Jordan, part of the fin de siecle fascination with the occult and (ironically enough given the background of Raiders' director), a French Catholic revival.
I can now also appreciate some of the finer edges that make this film just so much damn fun to watch. For instance, the comic book conceit that Jones and Belloq are a 'shadowy reflection' of one other and that they need 'only a nudge' to make one like the other. As is the idea that Belloq seems to be actually enjoying being evil. I also spotted Jones rejecting the martini on the plane. This feels less like a comment on corrupting effect of alcohol (he drinks rather heavily elsewhere in the film), but rather a subtle distancing from James Bond. This feels especially important given that the action-packed cold open is, let us be honest for a second, ripped straight from the 007 franchise.
John William's soundtracks are always worth mentioning. The corny Raiders March does almost nothing for me, but the highly-underrated 'Ark theme' certainly does. I delight in its allusions to Gregorian chant, the diabolus in musica and the Hungarian minor scale, fusing the Christian doctrine of the Holy Trinity (the stacked thirds, get it?), the ars antiqua of the Middle Ages with an 'exotic' twist that the Russian Five associated with central European Judaism.
The best use of the ark leitmotif is, of course, when it is opened. Here, Indy and Marion are saved by not opening their eyes whilst the 'High Priest' Belloq and the rest of the Nazis are all melted away. I'm no Biblical scholar, but I'm almost certain they were alluding to Leviticus 16:2 here:
The Lord said to Moses: Tell your brother Aaron that he is not to come whenever he chooses into the Most Holy Place behind the curtain in front of the atonement cover on the ark, or else he will die, for I will appear in the cloud above the mercy seat.
But would it be too much of a stretch to also see the myth of Orpheus and Eurydices too? Orpheus's wife would only be saved from the underworld if he did not turn around until he came to his own house. But he turned round to look at his wife, and she instantly slipped back into the depths:
For he who overcome should turn back his gaze
Towards the Tartarean cave,
Whatever excellence he takes with him
He loses when he looks on those below.
Perhaps not, given that Marion and the ark are not lost in quite the same way. But whilst touching on gender, it was interesting to update my view of archaeologist Ren Belloq. To countermand his slight queer coding (a trope of Disney villains such as Scar, Jafar, Cruella, etc.), there is a rather clumsy subplot involving Belloq repeatedly (and half-heartedly) failing to seduce Marion. This disavows any idea that Belloq isn't firmly heterosexual, essential for the film's mainstream audience, but it is especially important in Raiders because, if we recall the relationship between Belloq and Jones: 'it would take only a nudge to make you like me'. (This would definitely put a new slant on 'Top men'.)
However, my favourite moment is where the Nazis place the ark in a crate in order to transport it to the deserted island. On route, the swastikas on the side of the crate spontaneously burn away, and a disturbing noise is heard in the background. This short scene has always fascinated me, partly because it's the first time in the film that the power of the ark is demonstrated first-hand but also because gives the object an other-worldly nature that, to the best of my knowledge, has no parallel in the rest of cinema.
Still, I had always assumed that the Aak disfigured the swastikas because of their association with the Nazis, interpreting the act as God's condemnation of the Third Reich. But now I catch myself wondering whether the ark would have disfigured any iconography as a matter of principle or whether their treatment was specific to the swastika. We later get a partial answer to this question, as the 'US Army' inscriptions in the Citizen Kane warehouse remain untouched.
Far from being an insignificant concern, the filmmakers appear to have wandered into a highly-contested theological debate. As in, if the burning of the swastika is God's moral judgement of the Nazi regime, then God is clearly both willing and able to intervene in human affairs. So why did he not, to put it mildly, prevent Auschwitz? From this perspective, Spielberg appears to be limbering up for some of the academic critiques surrounding Holocaust representations that will follow Schindler's List (1993).
Given my nostalgic and somewhat ironic attachment to Raiders, it will always be difficult for me to objectively appraise the film. Even so, it feels like it is underpinned by an earnest attempt to entertain the viewer, largely absent in the affected cynicism of contemporary cinema. And when considered in the totality of Hollywood's output, its tonal and technical flaws are not actually that bad or at least Marion's muddled characterisation and its breezy chauvinism (for example) clearly have far worse examples.
Perhaps the most remarkable thing about the film in 2021 is that it hasn't changed that much at all. It spawned one good sequel (The Last Crusade), one bad one (The Temple of Doom), and one hardly worth mentioning at all, yet these adventures haven't affected the original Raiders in any meaningful way. In fact, if anything has affected the original text it is, once again, George Lucas himself, as knowing the impending backlash around the Star Wars prequels adds an inadvertent paratext to all his earlier works.
Yet in a 1978 discussion prior to the creation of Raiders, you can get a keen sense of how Lucas' childlike enthusiasm will always result in something either extremely good or something extremely bad somehow no middle ground is quite possible. Yes, it's easy to rubbish his initial ideas 'We'll call him Indiana Smith! but hasn't Lucas actually captured the essence of a heroic 'Americana' here, and that the final result is simply a difference of degree, not kind?
"Again, we see there is nothing you can possess which I cannot take away."
The cinema was a rare and expensive treat in my youth, so I first came across Raiders of the Lost Ark by recording it from television onto a poor quality VHS. I only mention this as it meant I watched a slightly different film to the one intended, as my copy somehow missed off the first 10 minutes. For those not as intimately familiar with the film as me, this is just in time to see a Belloq demand Dr. Jones hand over the Peruvian head (see above), just in time to learn that Indy loathes snakes, and just in time to see the inadvertent reproduction of two Europeans squabbling over the spoils of a foreign land.
What this truncation did to my interpretation of the film (released thirty years ago today on June 19th 1981) is interesting to explore. Without Jones' physical and moral traits being demonstrated on-screen (as well as missing the weighing the gold head and the rollercoaster boulder scene), it actually made the idea of 'Indiana Jones' even more of a mythical archetype. The film wisely withholds Jones' backstory, but my directors cut deprived him of even more, and counterintuitively imbued him with even more of a legendary hue as the elision made his qualities an assumption beyond question. Indiana Jones, if you can excuse the clich , needed no introduction at all.
Good artists copy, great artists steal. And oh boy, does Raiders steal. I've watched this film about twenty times over the past two decades and it's now firmly entered into my personal canon. But watching it on its thirtieth anniversary was different not least because I could situate it in a broader cinematic context. For example, I now see the Gestapo officer in Major Strasser from Casablanca (1942), in fact just as I can with many of Raiders' other orientalist tendencies: not only in its breezy depictions of backwards sand people, but also of North Africa as an entrep t and playground for a certain kind of Western gangster. The opening as well, set in an equally reductionist pseudo-Peru, now feels like Werner Herzog's Aguirre, the Wrath of God (1972) but without, of course, any self-conscious colonial critique.
The imagery of the ark appears to be borrowed from James Tissot's The Ark Passes Over the Jordan, part of the fin de siecle fascination with the occult and (ironically enough given the background of Raiders' director), a French Catholic revival.
I can now also appreciate some of the finer edges that make this film just so much damn fun to watch. For instance, the comic book conceit that Jones and Belloq are a 'shadowy reflection' of one other and that they need 'only a nudge' to make one like the other. As is the idea that Belloq seems to be actually enjoying being evil. I also spotted Jones rejecting the martini on the plane. This feels less like a comment on corrupting effect of alcohol (he drinks rather heavily elsewhere in the film), but rather a subtle distancing from James Bond. This feels especially important given that the action-packed cold open is, let us be honest for a second, ripped straight from the 007 franchise.
John William's soundtracks are always worth mentioning. The corny Raiders March does almost nothing for me, but the highly-underrated 'Ark theme' certainly does. I delight in its allusions to Gregorian chant, the diabolus in musica and the Hungarian minor scale, fusing the Christian doctrine of the Holy Trinity (the stacked thirds, get it?), the ars antiqua of the Middle Ages with an 'exotic' twist that the Russian Five associated with central European Judaism.
The best use of the ark leitmotif is, of course, when it is opened. Here, Indy and Marion are saved by not opening their eyes whilst the 'High Priest' Belloq and the rest of the Nazis are all melted away. I'm no Biblical scholar, but I'm almost certain they were alluding to Leviticus 16:2 here:
The Lord said to Moses: Tell your brother Aaron that he is not to come whenever he chooses into the Most Holy Place behind the curtain in front of the atonement cover on the ark, or else he will die, for I will appear in the cloud above the mercy seat.
But would it be too much of a stretch to also see the myth of Orpheus and Eurydices too? Orpheus's wife would only be saved from the underworld if he did not turn around until he came to his own house. But he turned round to look at his wife, and she instantly slipped back into the depths:
For he who overcome should turn back his gaze
Towards the Tartarean cave,
Whatever excellence he takes with him
He loses when he looks on those below.
Perhaps not, given that Marion and the ark are not lost in quite the same way. But whilst touching on gender, it was interesting to update my view of archaeologist Ren Belloq. To countermand his slight queer coding (a trope of Disney villains such as Scar, Jafar, Cruella, etc.), there is a rather clumsy subplot involving Belloq repeatedly (and half-heartedly) failing to seduce Marion. This disavows any idea that Belloq isn't firmly heterosexual, essential for the film's mainstream audience, but it is especially important in Raiders because, if we recall the relationship between Belloq and Jones: 'it would take only a nudge to make you like me'. (This would definitely put a new slant on 'Top men'.)
However, my favourite moment is where the Nazis place the ark in a crate in order to transport it to the deserted island. On route, the swastikas on the side of the crate spontaneously burn away, and a disturbing noise is heard in the background. This short scene has always fascinated me, partly because it's the first time in the film that the power of the ark is demonstrated first-hand but also because gives the object an other-worldly nature that, to the best of my knowledge, has no parallel in the rest of cinema.
Still, I had always assumed that the Aak disfigured the swastikas because of their association with the Nazis, interpreting the act as God's condemnation of the Third Reich. But now I catch myself wondering whether the ark would have disfigured any iconography as a matter of principle or whether their treatment was specific to the swastika. We later get a partial answer to this question, as the 'US Army' inscriptions in the Citizen Kane warehouse remain untouched.
Far from being an insignificant concern, the filmmakers appear to have wandered into a highly-contested theological debate. As in, if the burning of the swastika is God's moral judgement of the Nazi regime, then God is clearly both willing and able to intervene in human affairs. So why did he not, to put it mildly, prevent Auschwitz? From this perspective, Spielberg appears to be limbering up for some of the academic critiques surrounding Holocaust representations that will follow Schindler's List (1993).
Given my nostalgic and somewhat ironic attachment to Raiders, it will always be difficult for me to objectively appraise the film. Even so, it feels like it is underpinned by an earnest attempt to entertain the viewer, largely absent in the affected cynicism of contemporary cinema. And when considered in the totality of Hollywood's output, its tonal and technical flaws are not actually that bad or at least Marion's muddled characterisation and its breezy chauvinism (for example) clearly have far worse examples.
Perhaps the most remarkable thing about the film in 2021 is that it hasn't changed that much at all. It spawned one good sequel (The Last Crusade), one bad one (The Temple of Doom), and one hardly worth mentioning at all, yet these adventures haven't affected the original Raiders in any meaningful way. In fact, if anything has affected the original text it is, once again, George Lucas himself, as knowing the impending backlash around the Star Wars prequels adds an inadvertent paratext to all his earlier works.
Yet in a 1978 discussion prior to the creation of Raiders, you can get a keen sense of how Lucas' childlike enthusiasm will always result in something either extremely good or something extremely bad somehow no middle ground is quite possible. Yes, it's easy to rubbish his initial ideas 'We'll call him Indiana Smith! but hasn't Lucas actually captured the essence of a heroic 'Americana' here, and that the final result is simply a difference of degree, not kind?
8.5 years ago, I moved my
blog to
Ikiwiki and
Branchable. It's now time for me to take the
next step and host my blog on my own server. This is how I migrated from
Branchable to my own Apache server.
Installing Ikiwiki dependencies
Here are all of the extra Debian packages I had to install on my server:
and un-commented the following in /etc/apache2/mods-available/mime.conf:
AddHandler cgi-script .cgi
Creating a separate user account
Since Ikiwiki needs to regenerate my blog whenever a new article is pushed
to the git repo or a comment is accepted, I created a restricted user
account for it:
adduser blog
adduser blog sshuser
chsh -s /usr/bin/git-shell blog
git setup
Thanks to Branchable storing blogs in git repositories, I was able to import my
blog using a simple git clone in /home/blog (the srcdir):
Note that the name of the directory (source.git) is important for the
ikiwikihosting plugin to work.
Then I pulled the .setup file out of the setup branch in that repo and put
it in /home/blog/.ikiwiki/FeedingTheCloud.setup. After that, I deleted the
setup branch and the origin remote from that clone:
git branch -d setup
git remote rm origin
Following the recommended git
configuration, I created a working directory
(the repository) for the blog user to modify the blog as needed:
cd /home/blog/
git clone /home/blog/source.git FeedingTheCloud
I added my own ssh public key to /home/blog/.ssh/authorized_keys
so that I could push to the srcdir from my laptop.
Finaly, I generated a new ssh key without a passphrase:
One thing that failed to generate properly was the tag cloug (from the
pagestats plugin). I have not
been able to figure out why it fails to generate any output when run this
way, but if I push to the repo and let the git hook handle the rebuilding of
the wiki, the tag cloud is generated correctly. Consequently, fixing this
is not high on my list of priorities, but if you happen to know what the
problem is, please reach out.
Apache config
Here's the Apache config I put in /etc/apache2/sites-available/blog.conf:
a2ensite blog
apache2ctl configtest
systemctl restart apache2.service
The feeds.cloud.geek.nz domain used to be pointing to
Feedburner and so I need to
maintain it in order to avoid breaking RSS feeds from folks who added my
blog to their reader a long time ago.
Server-side improvements
Since I'm now in control of the server configuration, I was able to make
several improvements to how my blog is served.
First of all, I enabled the HTTP/2 and Brotli modules:
a2enmod http2
a2enmod brotli
and enabled Brotli
compression
by putting the following in /etc/apache2/conf-available/compression.conf:
Note that the Mozilla Observatory is mistakenly identifying HTTP onion
services as insecure, so you can ignore that failure.
I also used the Mozilla TLS config
generator
to improve the TLS config for my server.
Then I added security.txt and
gpc.json to the root
of my git repo and then added the following aliases to put these files in
the right place:
Alias /.well-known/gpc.json /var/www/blog/gpc.json
Alias /.well-known/security.txt /var/www/blog/security.txt
Monitoring 404s
Another advantage of running my own web server is that I can monitor the
404s easily using logcheck by
putting the following in /etc/logcheck/logcheck.logfiles:
/var/log/apache2/blog-error.log
Based on that, I added a few redirects to point bots and users to the
location of my RSS feed:
Future improvements
There are a few things I'd like to improve on my current setup.
The first one is to remove the iwikihosting and gitpush
plugins and replace them with a
small script which would simply git push to the read-only GitHub mirror.
Then I could uninstall the ikiwiki-hosting-common and
ikiwiki-hosting-web
since that's all I use them for.
Next, I would like to have proper support for signed git
pushes. At the
moment, I have the following in /home/blog/source.git/config:
but I'd like to also reject unsigned pushes.
While my blog now has a CSP policy which doesn't rely on unsafe-inline for
scripts, it does rely on unsafe-inline for stylesheets. I tried to remove this
but the actual calls to allow seemed to be located deep within jQuery and so I gave up.
Update: now fixed.
Finally, I'd like to figure out a good way to deal with articles which don't
currently have comments. At the moment, if you try to subscribe to their
comment feed, it returns a 404. For example:
[Sun Jun 06 17:43:12.336350 2021] [core:info] [pid 30591:tid 140253834704640] [client 66.249.66.70:57381] AH00128: File does not exist: /var/www/blog/posts/using-iptables-with-network-manager/comments.atom
This is obviously not ideal since many feed readers will refuse to add a
feed which is currently not found even though it could become real in the future. If
you know of a way to fix this, please let me know.
Custom screen saver for XSecureLock
Now that UDP was working, I was able to get TCP to work using two PlutoSDRs,
which allowed me to run the cURL command I pasted in the first post (both
simultaneously listen and transmit on behalf of my TAP interface).
It s my hope that someone out there will be inspired to implement their own
Layer 1 and Layer 2 as a learning exercise, and gets the same sense of
gratification that I did! If you re reading this, and at a point
where you ve been able to send IP traffic over your own Layer 1 / Layer 2,
please get in touch! I d be thrilled to hear all about it. I d love to link
to any posts or examples you publish here!
This post was originally published in the 
Colson Whitehead's latest novel, 
A quick note to self to remind how I do backups of my Android device with rsync (and adb).
I have followed this guide: 
Tonight I m provisioning a new virtual machine at
This blog post is about my observations of some of the plants in my home garden.
While still a n00b on the subject, these notes are my observations and experiences over days, weeks and months. Thankfully, with the capability to take frequent pictures, it has been easy to do an assessment and generate a report of some of these amazing behaviors of plants, in an easy timeline order; all thanks to the EXIF data embedded.
This has very helpfully allowed me to record my, otherwise minor observations, into great detail; and make some sense out of it by correlating the data over time.
It is an emotional experience. You see, plants are amazing. When I sow a sapling, water it, feed it, watch it grow, prune it, medicate it, and what not; I build up affection towards it.
Though, at the same time, to me it is a strict relationship, not too attached; as in it doesn t hurt to uproot a plant if there is a good reason. But still, I find some sort of association to it.
With plants around, it feels I have a lot of lives around me. All prospering, communicating, sharing. And communicate they do. What is needed is just the right language to observe and absorb their signals and decipher what they are trying to say.
"Again, we see there is nothing you can possess which I cannot take away."
The imagery of the ark appears to be borrowed from James Tissot's The Ark Passes Over the Jordan, part of the fin de siecle fascination with the occult and (ironically enough given the background of Raiders' director), a French Catholic revival.
.png){kind=link}
_Manila_Skyline_-_panoramio.jpg){kind=link}