Changes in version 0.0.2 (2021-01-29)

• Added GitHub Actions CI via run.sh from r-ci
• Updated to upstream version 3.0.0 (plus newer commits)

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

• Signal forces you to use the Google Play store and Google Services: This isn't true any more. You can download the apk directly on a phone without any Google services and it works great. The app will alert you to new versions. Don't get me wrong: the Signal network still depends on Google services. And, we should be avoiding corporate technology and building our own infrastructure. However, in practice, Signal is an alternative to Whatsapp and Telegram - which not only use the same corporate services but are proprietary technology that is fully owned by powerful tech giants. Signal is still a non-profit organization with a vastly different mission.
• Signal's approach to privacy isn't perfect (the most common variation on this theme is that a state actor could monitor your outgoing communications and the incoming communications of the person you are communicating with and prove that you are communicating with each other). This criticism missed what makes Signal so important. The beauty of Signal is that it addresses the "woops!" moment most privacy activists had when Snowden's data trove become public: it provides mass privacy to stop mass surveillance. Prior to 2013, most tech/privacy activists were focused on the "targeted" individual approach to privacy, working hard to make sure our tools were as absolutely perfect as possible for the tiny percentage of people who know they are under surveillance. Very little effort went into getting them adopted on a mass scale. Criticizing Signal for not providing perfect privacy misses that fact that these things often are trade offs. This trade-off also applies to the first point - dependency on Google services makes installation far easier for suporting millions of people.

• Signal is a centralized app: this criticism often includes examples of Moxie (Signal's founder) refusing and actively discouraging attempts by others to write software that interacts with Signal. Signal is free software, which is a major improvement over most corporate technology. But since it's entirely controlled by one entity, it can be shutdown in a heart beat. And, if Signal changes direction, we cannot easily take the work we have all invested in learning signal and create our own version that reflects our values. This problem is in contrast to federated systems like email - where anyone can run their own email server and apply their own policies. If one email serveer is shutdown, you can move to another. I agree with this critique, but I think it's nuanced because of the trade offs. Having full control over the entire network and all the software provides a level of reliability and consistency that would not be possible with a federated protocol. And, we already have three different, fully viable federated chat protocols (see above). I'd rather have Signal be Signal and invest our energy on a federated chat system via the existing, well-developed alternatives. This opinion is tactical - and could change at any moment. I think there will come a time when we are going to tell the world to move from Signal to the best available federated protocol. But I'm not convinced we have a robust enough federated chat infrastructure to support that move.
• Signal forces you to use your phone number as an identifier: You can't get a Signal account without a phone number. And you generally can't get a phone number without revealing some aspect of your identity. That makes staying anoymous very difficult. There are reports of a new Signal feature making it possible to avoid revealing your phone number when communicating with others, but you would still need a phone number to get an account because a SMS or phone call confirmation is required.
• Signal isn't getting ahead of the curve on abuse: There's an interesting piece informed by former Signal staff people about the management's resistance to getting ahead of the curve when it comes to abuse. How would signal respond to reports of harrassment? What would signal do if it recognized facsists movements organizing on its platform? Any mass platform that is not planning for abuse is going to be in big trouble very soon.

• Added support for systemd notifications and socket activation. Use of more features provided by systemd, including more notifications, will come in future releases. Thanks to Marco d'Itri for this first systemd integration into INN.
• nnrpd now adapts the length of the DH parameter used during a DHE key exchange so as to comply with the security level OpenSSL 1.1.0 or later expects. Thanks to Michael Baeuerle for the bug report.
• cnfsstat now also returns information about retired CNFS buffers: buffers mentioned in cycbuff.conf as a cycbuff but not declared in a metacycbuff.
• Switch default innreport behaviour to the common practice of externalizing CSS into a separate file. Its name can be configured with the html_css_url parameter in innreport.conf. If this parameter is unset, the default innreport.css file name will be used and innreport will generate this CSS file for you. Previously generated reports are kept untouched, though, and will still contain inline CSS if you had not already set the html_css_url parameter in previous INN versions. Thanks to Richard Kettlewell for the patch.
• sm can now read and store any number of articles given in wire format on its standard input when both -s and -R are used. Only native format was previously possible. Thanks to Bo Lindbergh for the patch.
• Added new -a flag to B<rnews> to disallow, if needed, the use of additional unpackers from the rnews.libexec sub-directory of pathbin (as set in inn.conf); only rnews and cunbatch will then be recognized as valid batch commands.
• Added new -b flag to rnews to save rejected articles in the bad subdirectory of pathincoming (as set in inn.conf). Otherwise, rnews just logs and discards any articles that are rejected or cannot be parsed for some reason.
• Added new -d flag to rnews to log via syslog the message-ID and the Path header value of each article rejected as a duplicate.
• Added new --enable-hardening-flags configure-time option, enabled by default, to use hardening build flags like -fPIE and -fstack-protector-strong. This option can easily be disabled if the compiler or the platform does not support them well. More hardening build flags will eventually be added in future releases.

Makefile:583: arch/amd64/Makefile: No such file or directory
make: *** No rule to make target 'arch/amd64/Makefile'.  Stop.

Makefile:583: arch/amd64/Makefile: No such file or directory
make: *** No rule to make target 'arch/amd64/Makefile'.  Stop.

1. Your worth is your productivity.
2. You cannot trust your own feelings and limits.
3. There is always more you could be doing.

Background While Raspbian, ahem, Raspberry PI OS is mostly Debian, the biggest difference is the kernel, both in terms of code and packaging. The packaging is weird since it needs to deal with the fact that there s no bootloader per se, the firmware parses /boot/config.txt and depending on the setting of 64bit and/or kernel line, it loads a specific file. Normally, one of kernel7.img, kernel7l.img or kernel8.img. While this configuration file supports an initrd, it doesn t have a clean way to associate an initrd with a kernel, but rather you have to (like for the actual kernel) settle on a hard-coded initrd name. Due to this, the normal way of using an initrd doesn t work, and one has to do two things:

• enable building initrd s at all
• settle on the naming for the initrd
• ensure the initrd is updated correctly

There are quite a few of forum threads about this, but there s no official support for this. The best link I found was this Stack Exchange post, which goes most of the way, but fails at the third point above.

My trivial solution Instead of naming tricks the above post suggests, I settled on having a fixed name. It risks boot failure when the kernel architecture will change, which could be worked around with hard-coded kernel too, but I haven t done that yet. First, enable initrd creation/update in /etc/default/raspberrypi-kernel, like the post says. Uncomment INITRD=yes, but not the RPI_INITRD. This will enable creating/updating the initrd when the kernel package is installed and/or other packages trigger it via hooks. Second, naming: choose an initrd name. I have simply this in my config.txt:

initramfs initrd.img followkernel

So the value is fully hard-coded, and the actual work is done in the next part. Last, add an initramfs-hook in (e.g.) /etc/initramfs/post-update.d/rpi-initrd. Note that, by default (unless other packages have created it), the /etc/initramfs directory doesn t exist. It s not the confusingly-named /etc/initramfs-tools/ directory, which is related to building the initrd, but rather to doing things with the (already built) initrd. This directory is briefly explain in the Debian kernel-handbook guide. This is my contents:

#!/bin/bash

ABI="$1"
INITRD="$2"
BOOTDIR="$(dirname "$INITRD")"

# Note: the match below _must_ be synced with the boot kernel
if [[ "$ABI" == *-v8+ ]]; then
        echo "Building v8l+ image, updating top initrd"
        cp -p "$INITRD" "$BOOTDIR/initrd.img"
fi

This script seems to me much simpler, so in principle less chance of bugs, than all the renaming and editing config.txt I saw in that stack exchange post. As far as I know, all one needs to care about is to sync the ABI passed to this hook with the kernel one is running, and since there is only one kernel version installed at a time on Raspbian (as there s no versioning in the kernel names), this should work correctly. With this hook, the update also works correctly when packages trigger initrd updates, not only when new kernels are installed. Note that the cp there is needed since the book partition is FAT, so no symbolic links/hard links allowed. Happy initrd ing!

How our for-profit company became a nonprofit, to better tackle the digital divide. Originally posted on the Endless OS Foundation blog.

An 8-year journey to a nonprofit On the 1st of April 2020, our for-profit Endless Mobile officially became a nonprofit as the Endless OS Foundation. Our launch as a nonprofit just as the global pandemic took hold was, predictably, hardly noticed, but for us the timing was incredible: as the world collectively asked What can we do to help others in need? , we framed our mission statement and launched our .org with the same very important question in mind. Endless always had a social impact mission at its heart, and the challenges related to students, families, and communities falling further into the digital divide during COVID-19 brought new urgency and purpose to our team s decision to officially step in the social welfare space.

On April 1st 2020, our for-profit Endless Mobile officially became a nonprofit as the Endless OS Foundation, focused on the #DigitalDivide.

Our updated status was a long time coming: we began our transformation to a nonprofit organization in late 2019 with the realization that the true charter and passions of our team would be greatly accelerated without the constraints of for-profit goals, investors and sales strategies standing in the way of our mission of digital access and equity for all. But for 8 years we made a go of it commercially, headquartered in Silicon Valley and framing ourselves as a tech startup with access to the venture capital and partnerships on our doorstep. We believed that a successful commercial channel would be the most efficient way to scale the impact of bringing computer devices and access to communities in need. We still believe this we ve just learned through our experience that we don t have the funding to enter the computer and OS marketplace head-on. With the social impact goal first, and the hope of any revenue a secondary goal, we have had many successes in those 8 years bridging the digital divide throughout the world, from Brazil, to Kenya, and the USA. We ve learned a huge amount which will go on to inform our strategy as a nonprofit.

Endless always had a social impact mission at its heart. COVID-19 brought new urgency and purpose to our team s decision to officially step in the social welfare space.

Our unique perspective One thing we learned as a for-profit is that the OS and technology we ve built has some unique properties which are hugely impactful as a working solution to digital equity barriers. And our experience deploying in the field around the world for 8 years has left us uniquely informed via many iterations and incremental improvements.

With this knowledge in-hand, we ve been refining our strategy throughout 2020 and now starting to focus on what it really means to become an effective nonprofit and make that impact. In many ways it is liberating to abandon the goals and constraints of being a for-profit entity, and in other ways it s been a challenging journey for me and the team to adjust our way of thinking and let these for-profit notions and models go. Previously we exclusively built and sold a product that defined our success; and any impact we achieved was a secondary consequence of that success and seen through that lens. Now our success is defined purely in terms of social impact, and through our actions, those positive impacts can be made with or without our product . That means that we may develop and introduce technology to solve a problem, but it is equally as valid to find another organization s existing offering and design a way to increase that positive impact and scale.

We develop technology to solve access equity issues, but it s equally as valid to find another organization s offering and partner in a way that increases their positive impact.

The analogy to Free and Open Source Software is very strong while Endless has always used and contributed to a wide variety of FOSS projects, we ve also had a tension where we ve been trying to hold some pieces back and capture value such as our own application or content ecosystem, our own hardware platform necessarily making us competitors to other organisations even though they were hoping to achieve the same things as us. As a nonprofit we can let these ideas go and just pick the best partners and technologies to help the people we re trying to reach.

Digital equity 4 barriers we need to overcome In future, our decisions around which projects to build or engage with will revolve around 4 barriers to digital equity, and how our Endless OS, Endless projects, or our partners offerings can help to solve them. We define these 4 equity barriers as: barriers to devices, barriers to connectivity, barriers to literacy in terms of your ability to use the technology, and barriers to engagement in terms of whether using the system is rewarding and worthwhile.

We define the 4 digital equity barriers we exist to impact as:
1. barriers to devices
2. barriers to connectivity
3. barriers to literacy
4. barriers to engagement

It doesn t matter who makes the solutions that break these barriers; what matters is how we assist in enabling people to use technology to gain access to the education and opportunities these barriers block. Our goal therefore is to simply ensure that solutions exist building them ourselves and with partners such as the FOSS community and other nonprofits proving them with real-world deployments, and sharing our results as widely as possible to allow for better adoption globally. If we define our goal purely in terms of whether people are using Endless OS, we are effectively restricting the reach and scale of our solutions to the audience we can reach directly with Endless OS downloads, installs and propagation. Conversely, partnerships that scale impact are a win-win-win for us, our partners, and the communities we all serve. Engineering impact Our Endless engineering roots and capabilities feed our unique ability to build and deploy all of our solutions, and the practical experience of deploying them gives us evidence and credibility as we advocate for their use. Either activity would be weaker without the other.

Our engineering roots and capabilities feed our unique ability to build and deploy digital divide solutions.

Our partners in various engineering communities will have already seen our change in approach. Particularly, with GNOME we are working hard to invest in upstream and reconcile the long-standing differences between our experience and GNOME. If successful, many more people can benefit from our work than just users of Endless OS. We re working with Learning Equality on Kolibri to build a better app experience for Linux desktop users and bring content publishers into our ecosystem for the first time, and we ve also taken our very own Hack, the immersive and fun destination for kids learning to code, released it for non-Endless systems on Flathub, and made it fully open-source.

What s next for our OS? What then is in store for the future of Endless OS, the place where we have invested so much time and planning through years of iterations? For the immediate future, we need the capacity to deploy everything we ve built all at once, to our partners. We built an OS that we feel is very unique and valuable, containing a number of world-firsts: first production OS shipped with OSTree, first Flatpak-only desktop, built-in support for updating OS and apps from USBs, while still providing a great deal of reliability and convenience for deployments in offline and educational-safe environments with great apps and content loaded on every system. However, we need to find a way to deliver this Linux-based experience in a more efficient way, and we d love to talk if you have ideas about how we can do this, perhaps as partners. Can the idea of Endless OS evolve to become a spec that is provided by different platforms in the future, maybe remixes of Debian, Fedora, openSUSE or Ubuntu? Build, Validate, Advocate Beyond the OS, the Endless OS Foundation has identified multiple programs to help underserved communities, and in each case we are adopting our build, validate, advocate strategy. This approach underpins all of our projects: can we build the technology (or assist in the making), will a community in-need validate it by adoption, and can we inspire others by telling the story and advocating for its wider use?

We are adopting a build, validate, advocate strategy.
1. build the technology (or assist in the making)
2. validate by community adoption
3. advocate for its wider use

As examples, we have just launched the Endless Key (link) as an offline solution for students during the COVID-19 at-home distance learning challenges. This project is also establishing a first-ever partnership of well-known online educational brands to reach an underserved offline audience with valuable learning resources. We are developing a pay-as-you-go platform and new partnerships that will allow families to own laptops via micro-payments that are built directly into the operating system, even if they cannot qualify for standard retail financing. And during the pandemic, we ve partnered with Teach For America to focus on very practical digital equity needs in the USA s urban and rural communities. One part of the world-wide digital divide solution We are one solution provider for the complex matrix of issues known collectively as the #DigitalDivide, and these issues will not disappear after the pandemic. Digital equity was an issue long before COVID-19, and we are not so naive to think it can be solved by any single institution, or by the time the pandemic recedes. It will take time and a coalition of partnerships to win. We are in for the long-haul and we are always looking for partners, especially now as we are finding our feet in the nonprofit world. We d love to hear from you, so please feel free to reach out to me I m ramcq on IRC, RocketChat, Twitter, LinkedIn or rob@endlessos.org.

• Timo R hling
• Fabio Augusto De Muzio Tobich
• Arun Kumar Pariyar
• Francis Murtagh
• William Desportes
• Robin Gustafsson
• Nicholas Guriev

usage: nspawn-runner [-h] [-v] [--debug]
                      chroot-create,chroot-login,prepare,run,cleanup,gitlab-config,toml 
                     ...
Manage systemd-nspawn machines for CI runs.
positional arguments:
   chroot-create,chroot-login,prepare,run,cleanup,gitlab-config,toml 
                        sub-command help
    chroot-create       create a chroot that serves as a base for ephemeral
                        machines
    chroot-login        enter the chroot to perform maintenance
    prepare             start an ephemeral system for a CI run
    run                 run a command inside a CI machine
    cleanup             cleanup a CI machine after it's run
    gitlab-config       configuration step for gitlab-runner
    toml                output the toml configuration for the custom runner
optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         verbose output
  --debug               verbose output

$ ./nspawn-runner toml buster
[[runners]]
  name="buster"
  url="TODO"
  token="TODO"
  executor = "custom"
  builds_dir = "/var/lib/nspawn-runner/.build"
  cache_dir = "/var/lib/nspawn-runner/.cache"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.custom]
    config_exec = "/home/enrico/ /nspawn-runner/nspawn-runner"
    config_args = ["gitlab-config"]
    config_exec_timeout = 200
    prepare_exec = "/home/enrico/ /nspawn-runner/nspawn-runner"
    prepare_args = ["prepare", "buster"]
    prepare_exec_timeout = 200
    run_exec = "/home/enrico/dev/nspawn-runner/nspawn-runner"
    run_args = ["run"]
    cleanup_exec = "/home/enrico/ /nspawn-runner/nspawn-runner"
    cleanup_args = ["cleanup"]
    cleanup_exec_timeout = 200
    graceful_kill_timeout = 200
    force_kill_timeout = 200

• updating the master chroot would disturb the running CI jobs that use it. Using nspawn's btrfs-specfic features would prevent this problem, and possibly simplify the implementation even more.
• New step! trivially implementing support for multiple OS images

• config can be a noop
• prepare starts the nspawn machine
• run runs scripts with machinectl shell
• cleanup runs machinectl stop

MACHINE="run-$CUSTOM_ENV_CI_JOB_ID"
ROOTFS="/var/lib/gitlab-runner-custom-chroots/buster"
OVERLAY="/var/lib/gitlab-runner-custom-chroots/$MACHINE"

#!/bin/sh
exit 0

#!/bin/bash
source $(dirname "$0")/base.sh
set -eo pipefail
# trap errors as a CI system failure
trap "exit $SYSTEM_FAILURE_EXIT_CODE" ERR
logger "gitlab CI: preparing $MACHINE"
mkdir -p $OVERLAY
systemd-run \
  -p 'KillMode=mixed' \
  -p 'Type=notify' \
  -p 'RestartForceExitStatus=133' \
  -p 'SuccessExitStatus=133' \
  -p 'Slice=machine.slice' \
  -p 'Delegate=yes' \
  -p 'TasksMax=16384' \
  -p 'WatchdogSec=3min' \
  systemd-nspawn --quiet -D $ROOTFS \
    --overlay="$ROOTFS:$OVERLAY:/"
    --machine="$MACHINE" --boot --notify-ready=yes

#!/bin/bash
logger "gitlab CI: running $@"
source $(dirname "$0")/base.sh
set -eo pipefail
trap "exit $SYSTEM_FAILURE_EXIT_CODE" ERR
systemd-run --quiet --pipe --wait --machine="$MACHINE" /bin/bash < "$1"

#!/bin/bash
logger "gitlab CI: cleanup $@"
source $(dirname "$0")/base.sh
machinectl stop "$MACHINE"
rm -rf $OVERLAY

# mkdir /var/lib/gitlab-runner-custom-chroots/build/
# mkdir /var/lib/gitlab-runner-custom-chroots/cache/
# gitlab-runner exec custom \
    --builds-dir /var/lib/gitlab-runner-custom-chroots/build/ \
    --cache-dir /var/lib/gitlab-runner-custom-chroots/cache/ \
    --custom-config-exec /var/lib/gitlab-runner-custom-chroots/config.sh \
    --custom-prepare-exec /var/lib/gitlab-runner-custom-chroots/prepare.sh \
    --custom-run-exec /var/lib/gitlab-runner-custom-chroots/run.sh \
    --custom-cleanup-exec /var/lib/gitlab-runner-custom-chroots/cleanup.sh \
    tests
Runtime platform                                    arch=amd64 os=linux pid=18662 revision=775dd39d version=13.8.0
Running with gitlab-runner 13.8.0 (775dd39d)
Preparing the "custom" executor
Using Custom executor...
Running as unit: run-r1be98e274224456184cbdefc0690bc71.service
executor not supported                              job=1 project=0 referee=metrics
Preparing environment
Getting source from Git repository
Executing "step_script" stage of the job script
WARNING: Starting with version 14.0 the 'build_script' stage will be replaced with 'step_script': https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26426
Job succeeded

concurrent = 1
check_interval = 0
[session_server]
  session_timeout = 1800
[[runners]]
  name = "nspawn runner"
  url = "http://gitlab.siweb.local/"
  token = " "
  executor = "custom"
  builds_dir = "/var/lib/gitlab-runner-custom-chroots/build/"
  cache_dir = "/var/lib/gitlab-runner-custom-chroots/cache/"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.custom]
    config_exec = "/var/lib/gitlab-runner-custom-chroots/config.sh"
    config_exec_timeout = 200
    prepare_exec = "/var/lib/gitlab-runner-custom-chroots/prepare.sh"
    prepare_exec_timeout = 200
    run_exec = "/var/lib/gitlab-runner-custom-chroots/run.sh"
    cleanup_exec = "/var/lib/gitlab-runner-custom-chroots/cleanup.sh"
    cleanup_exec_timeout = 200
    graceful_kill_timeout = 200
    force_kill_timeout = 200

mkdir test
cd test
git init
cat > .gitlab-ci.yml << EOF
tests:
 script:
  - env   sort
  - pwd
  - ls -la
EOF
git add .gitlab-ci.yml
git commit -am "Created a test repo for gitlab-runner"

gitlab-runner exec shell tests

• config, to allow to override the run configuration outputting JSON data
• prepare, to prepare the environment
• run, to run scripts in the environment (might be ran multiple times)
• cleanup to clean up the environment