As a lot of people do, I have some content that is reachable using
webbrowsers. There is the password manager
Vaultwarden, an instance
of Immich, ForgeJo for
some personal git repos, my blog and some other random pages here and
there.
All of this never had been a problem, running a webserver is a
relatively simple task, no matter if you use apache2 , nginx or any of the other possibilities. And the things
mentioned above bring their own daemon to serve the users.
AI crap
And then some idiot somewhere had the idea to ignore every law, every
copyright and every normal behaviour and run some shit AI bot. And
more idiots followed. And now we have more AI bots than humans
generating traffic.
And those AI shit crawlers do not respect any limits. robots.txt, slow
servers, anything to keep your meager little site up and alive? Them
idiots throw more resources onto them to steal content. No sense at
all.
iocaine to the rescue
So them AI bros want to ignore everything and just fetch the whole
internet? Without any consideration if thats even wanted? Or legal?
There are people who dislike this. I am one of them, but there are
some who got annoyed enough to develop tools to fight the AI
craziness. One of those tools is
iocaine - it says about
itself that it is The deadliest poison known to AI.
Feed AI bots sh*t
So you want content? You do not accept any Go away? Then here is
content. It is crap, but appearently you don t care. So have fun.
What iocaine does is (cite from their webpage) not made for making
the Crawlers go away. It is an aggressive defense mechanism that tries
its best to take the blunt of the assault, serve them garbage, and
keep them off of upstream resources .
That is, instead of the expensive webapp using a lot of resources that
are basically wasted for nothing, iocaine generates a small static
page (with some links back to itself, so the crawler shit stays
happy). Which takes a hell of a lot less resource than any fullblown
app.
iocaine setup
The website has a
https://iocaine.madhouse-project.org/documentation/, it is not hard to setup. Still, I had to adjust some
things for my setup, as I use [Caddy Docker
Proxy ([https://github.com/lucaslorentz/caddy-docker-proxy) nowadays
and wanted to keep the config within the docker setup, that is, within
the labels.
Caddy container
So my container setup for the caddy itself contains the following
extra lines:
labels:caddy_0.email:email@example.comcaddy_1:(iocaine)caddy_1.0_@read:method GET HEADcaddy_1.1_reverse_proxy:"@readiocaine:42069""caddy_1.1_reverse_proxy.@fallback":"status421"caddy_1.1_reverse_proxy.handle_response:"@fallback"
This will be translated to the following Caddy config snippet:
Any container that should be protected by iocaine
All the containers that are behind the Caddy reverse proxy can now
get protected by iocaine with just one more line in their
docker-compose.yaml. So now we have
So with one simple extra label for the docker container I have iocaine
activated.
Result? ByeBye (most) AI Bots
Looking at the services that got hammered most from those crap bots -
deploying this iocaine container and telling Caddy about it solved the
problem for me. 98% of the requests from the bots now go to iocaine
and no longer hog resources in the actual services.
I wish it wouldn t be neccessary to run such tools. But as long as we
have shitheads doing the AI hype there is no hope. I wish they all
would end up in Jail for all their various stealing they do. And
someone with a little more brain left would set things up sensibly,
then the AI thing could maybe turn out something good and useful.
But currently it is all crap.
Debian LTS
This was my hundred-thirty-sixth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. During my allocated time I uploaded or worked on:
[DLA 4316-1] open-vm-tools security update to fix one CVE related to a local privilege escalation.
[DLA 4329-1] libfcgi security update to fix one CVE related to a heap-based buffer overflow via crafted nameLen or valueLen values in data to the IPC socket.
[DLA 4337-1] svgpp security update to fix one CVE related to a nullpointer reference.
[DLA 4336-1] sysstat security update to fix two CVEs related to a size_t overflow and a multiplication integer overflow.
[DLA 4343-1] raptor2 security update to fix two CVEs related to a heap-based buffer over-read and an integer underflow.
[DLA 4349-1] request-tracker4 security update to fix one CVE related to CSV injection via ticket values with special characters. The patch was prepared by Andrew Ruthven
[DLA 4353-1] xorg-server security update to fix three CVES related to privilege escalation.
I also attended the monthly LTS/ELTS meeting.
Debian ELTS
This month was the eighty-seventh ELTS month. During my allocated time I uploaded or worked on:
[ELA-1538-1] libfcgi security update to fix one CVE in Buster and Stretch, related to a heap-based buffer overflow via crafted nameLen or valueLen values in data to the IPC socket.
[ELA-1551-1] raptor2 security update to fix two CVES in Buster and Stretch, related to a heap-based buffer over-read and an integer underflow.
[ELA-1555-1] request-tracker4 security update to fix one CVE in Buster, related to CSV injection via ticket values with special characters. The patch was prepared by Andrew Ruthven.
[ELA-1561-1] xorg-server security update to fix three CVEs in Buster and Stretch, related to privilege escalation.
I also attended the monthly LTS/ELTS meeting.
Debian Printing
This month I uploaded a new upstream version or a bugfix version of:
Debian IoT
Unfortunately I didn t found any time to work on this topic.
Debian Mobcom
This month I uploaded a new upstream version or a bugfix version of:
On my fight against outdated RFPs, I closed 31 of them in October. I could even close one RFP by uploading the new package gypsy. Meanwhile only 3373 are still open, so don t hesitate to help closing one or another.
FTP master
This month I accepted 420 and rejected 45 packages. The overall number of packages that got accepted was 423.
I would like to remind everybody that in case you don t agree with the removal of a package, please set the moreinfo tag on this bug. This is the only reliable way to prevent processing of that RM-bug. Well, there is a second way, of course you could also achieve this by closing the bug.
Debian LTS
This was my hundred-thirty-fifth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. During my allocated time I uploaded or worked on:
[DLA 4168-2] openafs regression update to fix an incomplete patch in the previous upload.
[DSA 5998-1] cups security update to fix two CVEs related to a authentication bypass and a denial of service.
[DLA 4298-1] cups security update to fix two CVEs related to a authentication bypass and a denial of service.
[DLA 4304-1] cjson security update to fix one CVE related to an out-of-bounds memory access.
[DLA 4307-1] jq security update to fix one CVE related to a heap buffer overflow.
[DLA 4308-1] corosync security update to fix one CVE related to a stack-based buffer overflow.
An upload of spim was not needed, as the corresponding CVE could be marked as ignored.
I also started to work on an open-vm-tools and attended the monthly LTS/ELTS meeting.
Debian ELTS
This month was the eighty-sixth ELTS month. During my allocated time I uploaded or worked on:
[ELA-1512-1] cups security update to fix two CVEs in Buster and Stretch, related to a authentication bypass and a denial of service.
[ELA-1520-1] jq security update to fix one CVE in Buster and Stretch, related to a heap buffer overflow.
[ELA-1524-1] corosync security update to fix one CVE in Buster and Stretch, related to a stack-based buffer overflow.
[ELA-1527-1] mplayer security update to fix ten CVEs in Stretch, distributed all over the code.
The CVEs for open-vm-tools could be marked as not-affeceted as the corresponding plugin was not yet available. I also attended the monthly LTS/ELTS meeting.
Debian Printing
This month I uploaded a new upstream version or a bugfix version of:
misc
The main topic of this month has been gcc15 and cmake4, so my upload rate was extra high. This month I uploaded a new upstream version or a bugfix version of:
I wonder what MBF will happen next, I guess the /var/lock-issue will be a good candidate.
On my fight against outdated RFPs, I closed 30 of them in September. Meanwhile only 3397 are still open, so don t hesitate to help closing one or another.
FTP master
This month I accepted 294 and rejected 28 packages. The overall number of packages that got accepted was 294.
Utkarsh Gupta
did 1.0h (out of 15.0h assigned), thus carrying over 14.0h to the next month.
Evolution of the situation
In May, we released 54 DLAs.
The LTS Team was particularly active in May, publishing a higher than normal number of advisories, as well as helping with a wide range of updates to packages in stable and unstable, plus some other interesting work. We are also pleased to welcome several updates from contributors outside the regular team.
Notable security updates:
containerd, prepared by Andreas Henriksson, fixes a vulnerability that could cause containers launched as non-root users to be run as root
libapache2-mod-auth-openidc, prepared by Moritz Schlarb, fixes a vulnerability which could allow an attacker to crash an Apache web server with libapache2-mod-auth-openidc installed
request-tracker4, prepared by Andrew Ruthven, fixes multiple vulnerabilities which could result in information disclosure, cross-site scripting and use of weak encryption for S/MIME emails
postgresql-13, prepared by Bastien Roucari s, fixes an application crash vulnerability that could affect the server or applications using libpq
dropbear, prepared by Guilhem Moulin, fixes a vulnerability which could potentially result in execution of arbitrary shell commands
openjdk-17, openjdk-11, prepared by Thorsten Glaser, fixes several vulnerabilities, which include denial of service, information disclosure or bypass of sandbox restrictions
glibc, prepared by Sean Whitton, fixes a privilege escalation vulnerability
Notable non-security updates:
wireless-regdb, prepared by Ben Hutchings, updates information reflecting changes to radio regulations in many countries
This month s contributions from outside the regular team include the libapache2-mod-auth-openidc update mentioned above, prepared by Moritz Schlarb (the maintainer of the package); the update of request-tracker4, prepared by Andrew Ruthven (the maintainer of the package); and the updates of openjdk-17 and openjdk-11, also noted above, prepared by Thorsten Glaser.
Additionally, LTS Team members contributed stable updates of the following packages:
rubygems and yelp/yelp-xsl, prepared by Lucas Kanashiro
simplesamlphp, prepared by Tobias Frost
libbson-xs-perl, prepared by Roberto C. S nchez
fossil, prepared by Sylvain Beucler
setuptools and mydumper, prepared by Lee Garrett
redis and webpy, prepared by Adrian Bunk
xrdp, prepared by Abhijith PA
tcpdf, prepared by Santiago Ruano Rinc n
kmail-account-wizard, prepared by Thorsten Alteholz
Other contributions were also made by LTS Team members to packages in unstable:
proftpd-dfsg DEP-8 tests (autopkgtests) were provided to the maintainer, prepared by Lucas Kanashiro
a regular upload of libsoup2.4, prepared by Sean Whitton
a regular upload of setuptools, prepared by Lee Garrett
Freexian, the entity behind the management of the Debian LTS project, has been working for some time now on the development of an advanced CI platform for Debian-based distributions, called Debusine. Recently, Debusine has reached a level of feature implementation that makes it very usable. Some members of the LTS Team have been using Debusine informally, and during May LTS coordinator Santiago Ruano Rinc n has made a call for the team to help with testing of Debusine, and to help evaluate its suitability for the LTS Team to eventually begin using as the primary mechanism for uploading packages into Debian. Team members who have started using Debusine are providing valuable feedback to the Debusine development team, thus helping to improve the platform for all users. Actually, a number of updates, for both bullseye and bookworm, made during the month of May were handled using Debusine, e.g. rubygems s DLA-4163-1.
By the way, if you are a Debian Developer, you can easily test Debusine following the instructions found at https://wiki.debian.org/DebusineDebianNet.
DebConf, the annual Debian Conference, is coming up in July and, as is customary each year, the week preceding the conference will feature an event called DebCamp. The DebCamp week provides an opportunity for teams and other interested groups/individuals to meet together in person in the same venue as the conference itself, with the purpose of doing focused work, often called sprints . LTS coordinator Roberto C. S nchez has announced that the LTS Team is planning to hold a sprint primarily focused on the Debian security tracker and the associated tooling used by the LTS Team and the Debian Security Team.
Thanks to our sponsors
Sponsors that joined recently are in bold.
Evolution of the situation
In April, we released 46 DLAs.
Notable security updates:
jetty9, prepared by Markus Koschany, fixes an information disclosure and potential remote code execution vulnerability
zabbix, prepared by Tobias Frost, fixes several vulnerabilities, encompassing denial of service, information disclosure or remote code inclusion
glibc, prepared by Sean Whitton, fixes a buffer overflow vulnerability
Notable non-security updates:
tzdata, prepared by Emilio Pozuelo Monfort, brings the latest timezone database release
php-horde-editor and php-horde-imp, prepared by Sylvain Beucler, have been updated to switch from CKEditor v3, which is EOL, to CKEditor v4; this builds upon work done last month by Sylvain and Bastien for the complete removal of ckeditor3
distro-info-data, prepared by Stefano Rivera, adds information concerning future Debian and Ubuntu releases
The LTS team continues to welcome the collaboration of maintainers and other interested parties from outside the regular team. In April, we had external updates contributed by: Yadd - lemonldap-ng and Moritz Schlarb - libapache2-mod-auth-openidc
A point release of the current stable Debian 12 (codename bookworm ) is planned for mid-May and several LTS contributors have prepared packages for this update, many of them prepared in conjunction with related LTS updates of the same packages:
glib2.0, haproxy, imagemagick, poppler, and python-h11, prepared by Adrian Bunk
rubygems, prepared by Lucas Kanashiro
ruby3.1 (in collaboration with Lucas Kanashiro), twitter-bootstrap3, twitterboot-strap4, wpa, and erlang, prepared by Bastien Roucari s (corresponding updates of twitter-bootstrap3 and twitter-bootstrap4 were also uploaded to Debian unstable)
abseil, prepared by Tobias Frost (a corresponding update was also uploaded to Debian unstable)
vips, prepared by Guilhem Moulin
Additional updates of ruby3.3 and rubygems were prepared for Debian unstable by Lucas Kanashiro.
And finally, a highlight of our continued commitment to enhancing long term support efforts in upstream projects. Freexian, as the primary entity behind the management and execution of the LTS project, has partnered with Invisible Things Lab to extend the upstream security support of Xen 4.17, which is shipped in Debian 12 bookworm (the current stable release). This partnership will result in significantly improved lifecycle support for users of Xen on bookworm, and members of the LTS team will play a part in this endeavour. The Freexian announcement has additional details.
Thanks to our sponsors
Sponsors that joined recently are in bold.
Nextcloud is an open-source software suite that
enables you to set up and manage your own cloud storage and collaboration
platform. It offers a range of features similar to popular cloud services
like Google Drive or Dropbox but with the added benefit of complete control
over your data and the server where it s hosted.
I wanted to have a look at Nextcloud and the steps to setup a own instance
with a PostgreSQL based database together with NGinx as the webserver to
serve the WebUI. Before doing a full productive setup I wanted to play
around locally with all the needed steps and worked out all the steps within
KVM machine.
While doing this I wrote down some notes to mostly document for myself what
I need to do to get a Nextcloud installation running and usable. So this
manual describes how to setup a Nextcloud installation on Debian 12 Bookworm
based on NGinx and PostgreSQL.
Nextcloud Installation
Install PHP and PHP extensions for Nextcloud
Nextcloud is basically a PHP application so we need to install PHP packages
to get it working in the end. The following steps are based on the upstream
documentation about how to install a own
Nextcloud instance.
Installing the virtual package package php on a Debian Bookworm system
would pull in the depending meta package php8.2. This package itself would
then pull also the package libapache2-mod-php8.2 as an dependency which
then would pull in also the apache2 webserver as a depending package. This
is something I don t wanted to have as I want to use NGinx that is already
installed on the system instead.
To get this we need to explicitly exclude the package libapache2-mod-php8.2
from the list of packages which we want to install, to achieve this we have
to append a hyphen - at the end of the package name, so we need to use
libapache2-mod-php8.2- within the package list that is telling apt to
ignore this package as an dependency. I ended up with this call to get all
needed dependencies installed.
To make these settings effective, restart the php-fpm service
$ sudo systemctl restart php8.2-fpm
Install PostgreSQL, Create a database and user
This manual assumes we will use a PostgreSQL server on localhost, if you
have a server instance on some remote site you can skip the installation
step here.
$ sudo apt install postgresql postgresql-contrib postgresql-client
Check version after installation (optinal step):
$ sudo -i -u postgres$ psql -version
This output will be seen:
psql (15.12 (Debian 15.12-0+deb12u2))
Exit the PSQL shell by using the command \q.
postgres=# \q
Exit the CLI of the postgres user:
postgres@host:~$ exit
Create a PostgreSQL Database and User:
Create a new PostgreSQL user (Use a strong password!):
$ sudo -u postgres psql -c "CREATE USER nextcloud_user PASSWORD '1234';"
Create new database and grant access:
$ sudo -u postgres psql -c "CREATE DATABASE nextcloud_db WITH OWNER nextcloud_user ENCODING=UTF8;"
(Optional) Check if we now can connect to the database server and the
database in detail (you will get a question about the password for the database
user!). If this is not working it makes no sense to proceed further! We need to
fix first the access then!
$ psql -h localhost -U nextcloud_user -d nextcloud_db
or
$ psql -h 127.0.0.1 -U nextcloud_user -d nextcloud_db
Log out from postgres shell using the command \q.
Download and install Nextcloud
Use the following command to download the latest version of Nextcloud:
$ wget https://download.nextcloud.com/server/releases/latest.zip
Extract file into the folder /var/www/html with the following command:
$ sudo unzip latest.zip -d /var/www/html
Change ownership of the /var/www/html/nextcloud directory to www-data.
$ sudo chown -R www-data:www-data /var/www/html/nextcloud
Configure NGinx for Nextcloud to use a certificate
In case you want to use self signed certificate, e.g. if you play around to
setup Nextcloud locally for testing purposes you can do the following steps.
If you want or need to use the service of Let s Encrypt (or similar) drop
the step above and create your required key data by using this command:
$ sudo certbot --nginx -d nextcloud.your-domain.com
You will need to adjust the path to the key and certificate in the next
step!
Change the NGinx configuration:
$ sudo vi /etc/nginx/sites-available/nextcloud.conf
Add the following snippet into the file and save it.
# /etc/nginx/sites-available/nextcloud.conf
upstreamphp-handler#server 127.0.0.1:9000;
serverunix:/run/php/php8.2-fpm.sock;
# Set the immutable cache control options only for assets with a cache
# busting v argument
map $arg_v $asset_immutable
"""";
default",immutable";
serverlisten80;
listen[::]:80;
# Adjust this to the correct server name!
server_namenextcloud.local;
# Prevent NGinx HTTP Server Detection
server_tokensoff;
# Enforce HTTPS
return301https://$server_name$request_uri;
serverlisten443sslhttp2;
listen[::]:443sslhttp2;
# Adjust this to the correct server name!
server_namenextcloud.local;
# Path to the root of your installation
root/var/www/html/nextcloud;
# Use Mozilla's guidelines for SSL/TLS settings
# https://mozilla.github.io/server-side-tls/ssl-config-generator/
# Adjust the usage and paths of the correct key data! E.g. it you want to use Let's Encrypt key material!
ssl_certificate/etc/ssl/certs/nextcloud.crt;
ssl_certificate_key/etc/ssl/private/nextcloud.key;
# ssl_certificate /etc/letsencrypt/live/nextcloud.your-domain.com/fullchain.pem;
# ssl_certificate_key /etc/letsencrypt/live/nextcloud.your-domain.com/privkey.pem;
# Prevent NGinx HTTP Server Detection
server_tokensoff;
# HSTS settings
# WARNING: Only add the preload option once you read about
# the consequences in https://hstspreload.org/. This option
# will add the domain to a hardcoded list that is shipped
# in all major browsers and getting removed from this list
# could take several months.
#add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload" always;
# set max upload size and increase upload timeout:
client_max_body_size512M;
client_body_timeout300s;
fastcgi_buffers644K;
# Enable gzip but do not remove ETag headers
gzipon;
gzip_varyon;
gzip_comp_level4;
gzip_min_length256;
gzip_proxiedexpiredno-cacheno-storeprivateno_last_modifiedno_etagauth;
gzip_typesapplication/atom+xmltext/javascriptapplication/javascriptapplication/jsonapplication/ld+jsonapplication/manifest+jsonapplication/rss+xmlapplication/vnd.geo+jsonapplication/vnd.ms-fontobjectapplication/wasmapplication/x-font-ttfapplication/x-web-app-manifest+jsonapplication/xhtml+xmlapplication/xmlfont/opentypeimage/bmpimage/svg+xmlimage/x-icontext/cache-manifesttext/csstext/plaintext/vcardtext/vnd.rim.location.xloctext/vtttext/x-componenttext/x-cross-domain-policy;
# Pagespeed is not supported by Nextcloud, so if your server is built
# with the ngx_pagespeed module, uncomment this line to disable it.
#pagespeed off;
# The settings allows you to optimize the HTTP2 bandwidth.
# See https://blog.cloudflare.com/delivering-http-2-upload-speed-improvements/
# for tuning hints
client_body_buffer_size512k;
# HTTP response headers borrowed from Nextcloud .htaccess
add_headerReferrer-Policy"no-referrer"always;
add_headerX-Content-Type-Options"nosniff"always;
add_headerX-Frame-Options"SAMEORIGIN"always;
add_headerX-Permitted-Cross-Domain-Policies"none"always;
add_headerX-Robots-Tag"noindex,nofollow"always;
add_headerX-XSS-Protection"1; mode=block"always;
# Remove X-Powered-By, which is an information leak
fastcgi_hide_headerX-Powered-By;
# Set .mjs and .wasm MIME types
# Either include it in the default mime.types list
# and include that list explicitly or add the file extension
# only for Nextcloud like below:
includemime.types;
typestext/javascriptjsmjs;
application/wasmwasm;
# Specify how to handle directories -- specifying /index.php$request_uri
# here as the fallback means that NGinx always exhibits the desired behaviour
# when a client requests a path that corresponds to a directory that exists
# on the server. In particular, if that directory contains an index.php file,
# that file is correctly served; if it doesn't, then the request is passed to
# the front-end controller. This consistent behaviour means that we don't need
# to specify custom rules for certain paths (e.g. images and other assets,
# /updater , /ocs-provider ), and thus
# try_files $uri $uri/ /index.php$request_uri
# always provides the desired behaviour.
indexindex.phpindex.html/index.php$request_uri;
# Rule borrowed from .htaccess to handle Microsoft DAV clients
location = /if( $http_user_agent ~ ^DavClnt)return302/remote.php/webdav/$is_args$args;
location = /robots.txtallowall;
log_not_foundoff;
access_logoff;
# Make a regex exception for /.well-known so that clients can still
# access it despite the existence of the regex rule
# location ~ /(\. autotest ...) which would otherwise handle requests
# for /.well-known .
location^~/.well-known# The rules in this block are an adaptation of the rules
# in .htaccess that concern /.well-known .
location = /.well-known/carddavreturn301/remote.php/dav/;
location = /.well-known/caldavreturn301/remote.php/dav/;
location/.well-known/acme-challengetry_files $uri $uri/ =404;
location/.well-known/pki-validationtry_files $uri $uri/ =404;
# Let Nextcloud's API for /.well-known URIs handle all other
# requests by passing them to the front-end controller.
return301/index.php$request_uri;
# Rules borrowed from .htaccess to hide certain paths from clients
location ~ ^/(?:build tests config lib 3rdparty templates data)(?:$ /)return404;
location ~ ^/(?:\. autotest occ issue indie db_ console)return404;
# Ensure this block, which passes PHP files to the PHP process, is above the blocks
# which handle static assets (as seen below). If this block is not declared first,
# then NGinx will encounter an infinite rewriting loop when it prepend /index.php
# to the URI, resulting in a HTTP 500 error response.
location ~ \.php(?:$ /)# Required for legacy support
rewrite^/(?!index remote public cron core\/ajax\/update status ocs\/v[12] updater\/.+ ocs-provider\/.+ .+\/richdocumentscode(_arm64)?\/proxy)/index.php$request_uri;
fastcgi_split_path_info^(.+?\.php)(/.*)$;
set $path_info $fastcgi_path_info;
try_files $fastcgi_script_name =404;
includefastcgi_params;
fastcgi_paramSCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_paramPATH_INFO $path_info;
fastcgi_paramHTTPSon;
fastcgi_parammodHeadersAvailabletrue; # Avoid sending the security headers twice
fastcgi_paramfront_controller_activetrue; # Enable pretty urls
fastcgi_passphp-handler;
fastcgi_intercept_errorson;
fastcgi_request_bufferingoff;
fastcgi_max_temp_file_size0;
# Serve static files
location ~ \.(?:css js mjs svg gif png jpg ico wasm tflite map ogg flac)$try_files $uri /index.php$request_uri;
# HTTP response headers borrowed from Nextcloud .htaccess
add_headerCache-Control"public,max-age=15778463$asset_immutable";
add_headerReferrer-Policy"no-referrer"always;
add_headerX-Content-Type-Options"nosniff"always;
add_headerX-Frame-Options"SAMEORIGIN"always;
add_headerX-Permitted-Cross-Domain-Policies"none"always;
add_headerX-Robots-Tag"noindex,nofollow"always;
add_headerX-XSS-Protection"1; mode=block"always;
access_logoff; # Optional: Don't log access to assets
location ~ \.woff2?$try_files $uri /index.php$request_uri;
expires7d; # Cache-Control policy borrowed from .htaccess
access_logoff; # Optional: Don't log access to assets
# Rule borrowed from .htaccess
location/remotereturn301/remote.php$request_uri;
location/try_files $uri $uri/ /index.php$request_uri;
Symlink configuration site available to site enabled.
$ ln -s /etc/nginx/sites-available/nextcloud.conf /etc/nginx/sites-enabled/
Restart NGinx and access the URI in the browser.
Go through the installation of Nextcloud.
The user data on the installation dialog should point e.g to
administrator or similar, that user will become administrative access
rights in Nextcloud!
To adjust the database connection detail you have to edit the file
$install_folder/config/config.php.
Means here in the example within this post you would need to modify
/var/www/html/nextcloud/config/config.php to control or change the
database connection.
---%<---'dbname'=>'nextcloud_db',
'dbhost'=>'localhost', #(Or your remote PostgreSQL server address if you have.)
'dbport'=>'',
'dbtableprefix'=>'oc_',
'dbuser'=>'nextcloud_user',
'dbpassword'=>'1234', #(The password you set for database user.)
--->%---
After the installation and setup of the Nextcloud PHP application there are
more steps to be done. Have a look into the WebUI what you will need to do
as additional steps like create a cronjob or tuning of some more PHP
configurations.
If you ve done all things correct you should see a login page similar to
this:
Optional other steps for more enhanced configuration modifications
Move the data folder to somewhere else
The data folder is the root folder for all user content. By default it is
located in $install_folder/data, so in our case here it is in
/var/www/html/nextcloud/data.
Move the data directory outside the web server document root.
$ sudo mv /var/www/html/nextcloud/data /var/nextcloud_data
Ensure access permissions, mostly not needed if you move the folder.
$ sudo chown -R www-data:www-data /var/nextcloud_data$ sudo chown -R www-data:www-data /var/www/html/nextcloud/
Update the Nextcloud configuration:
Open the config/config.php file of your Nextcloud installation.
$ sudo vi /var/www/html/nextcloud/config/config.php
Update the datadirectory parameter to point to the new location of your data directory.
Make the installation available for multiple FQDNs on the same server
Adjust the Nextcloud configuration to listen and accept requests for
different domain names. Configure and adjust the key trusted_domains
accordingly.
$ sudo vi /var/www/html/nextcloud/config/config.php
Create and adjust the needed site configurations for the webserver.
Restart the NGinx unit.
An error message about .ocdata might occur
.ocdata is not found inside the data directory
Create file using touch and set necessary permissions.
$ sudo touch /var/nextcloud_data/.ocdata$ sudo chown -R www-data:www-data /var/nextcloud_data/
The password for the administrator user is unknown
Log in to your server:
SSH into the server where your PostgreSQL database is hosted.
Switch to the PostgreSQL user:
$ sudo -i -u postgres
Access the PostgreSQL command line
psql
List the databases: (If you re unsure which database is being used by Nextcloud, you can list all the databases by the list command.)
\l
Switch to the Nextcloud database:
Switch to the specific database that Nextcloud is using.
\c nextclouddb
Reset the password for the Nextcloud database user:
ALTER USER nextcloud_user WITH PASSWORD 'new_password';
Exit the PostgreSQL command line:
\q
Verify Database Configuration:
Check the database connection details in the config.php file to ensure they are correct.
sudo vi /var/www/html/nextcloud/config/config.php
Replace nextcloud_db, nextcloud_user, and your_password with your actual database name, user, and password.
---%<---'dbname'=>'nextcloud_db',
'dbhost'=>'localhost', #(or your PostgreSQL server address)
'dbport'=>'',
'dbtableprefix'=>'oc_',
'dbuser'=>'nextcloud_user',
'dbpassword'=>'1234', #(The password you set for nextcloud_user.)
--->%---
Restart NGinx and access the UI through the browser.
Welcome to the third report in 2025 from the Reproducible Builds project. Our monthly reports outline what we ve been up to over the past month, and highlight items of news from elsewhere in the increasingly-important area of software supply-chain security. As usual, however, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website.
Table of contents:
Debian bookworm live images now fully reproducible from their binary packages
Roland Clobus announced on our mailing list this month that all the major desktop variants (ie. Gnome, KDE, etc.) can be reproducibly created for Debian bullseye, bookworm and trixie from their (pre-compiled) binary packages.
Building reproducible Debian live images does not require building from reproducible source code, but this is still a remarkable achievement. Some large proportion of the binary packages that comprise these live images can (and were) built reproducibly, but live image generation works at a higher level. (By contrast, full or end-to-end reproducibility of a bootable OS image will, in time, require both the compile-the-packages the build-the-bootable-image stages to be reproducible.)
Nevertheless, in response, Roland s announcement generated significant congratulations as well as some discussion regarding the finer points of the terms employed: a full outline of the replies can be found here.
The news was also picked up by Linux Weekly News (LWN) as well as to Hacker News.
LWN: Fedora change aims for 99% package reproducibilityLinux Weekly News (LWN) contributor Joe Brockmeier has published a detailed round-up on how Fedora change aims for 99% package reproducibility. The article opens by mentioning that although Debian has been working toward reproducible builds for more than a decade , the Fedora project has now:
progressed far enough that the project is now considering a change proposal for the Fedora 43 development cycle, expected to be released in October, with a goal of making 99% of Fedora s package builds reproducible. So far, reaction to the proposal seems favorable and focused primarily on how to achieve the goal with minimal pain for packagers rather than whether to attempt it.
Over the last few releases, we [Fedora] changed our build infrastructure to make package builds reproducible. This is enough to reach 90%. The remaining issues need to be fixed in individual packages. After this Change, package builds are expected to be reproducible. Bugs will be filed against packages when an irreproducibility is detected. The goal is to have no fewer than 99% of package builds reproducible.
Python adopts PEP standard for specifying package dependencies
Python developer Brett Cannonreported on Fosstodon that PEP 751 was recently accepted. This design document has the purpose of describing a file format to record Python dependencies for installation reproducibility . As the abstract of the proposal writes:
This PEP proposes a new file format for specifying dependencies to enable reproducible installation in a Python environment. The format is designed to be human-readable and machine-generated. Installers consuming the file should be able to calculate what to install without the need for dependency resolution at install-time.
The PEP, which itself supersedes PEP 665, mentions that there are at least five well-known solutions to this problem in the community .
OSS Rebuild real-time validation and tooling improvements
OSS Rebuild aims to automate rebuilding upstream language packages (e.g. from PyPI, crates.io, npm registries) and publish signed attestations and build definitions for public use.
OSS Rebuild is now attempting rebuilds as packages are published, shortening the time to validating rebuilds and publishing attestations.
Aman Sharma contributed classifiers and fixes for common sources of non-determinism in JAR packages.
Improvements were also made to some of the core tools in the project:
timewarp for simulating the registry responses from sometime in the past.
proxy for transparent interception and logging of network activity.
SimpleX Chat server components now reproducible
SimpleX Chat is a privacy-oriented decentralised messaging platform that eliminates user identifiers and metadata, offers end-to-end encryption and has a unique approach to decentralised identity. Starting from version 6.3, however, Simplex has implemented reproducible builds for its server components. This advancement allows anyone to verify that the binaries distributed by SimpleX match the source code, improving transparency and trustworthiness.
Three new scholarly papers
Aman Sharma of the KTH Royal Institute of Technology of Stockholm, Sweden published a paper on Build and Runtime Integrity for Java (PDF). The paper s abstract notes that Software Supply Chain attacks are increasingly threatening the security of software systems and goes on to compare build- and run-time integrity:
Build-time integrity ensures that the software artifact creation process, from source code to compiled binaries, remains untampered. Runtime integrity, on the other hand, guarantees that the executing application loads and runs only
trusted code, preventing dynamic injection of malicious components.
The recently mandated software bill of materials (SBOM) is intended to help mitigate software supply-chain risk. We discuss extensions that would enable an SBOM to serve as a basis for making trust assessments thus also serving as a proactive defense.
A full PDF of the paper is available.
Lastly, congratulations to Giacomo Benedetti of the University of Genoa for publishing their PhD thesis. Titled Improving Transparency, Trust, and Automation in the Software Supply Chain, Giacomo s thesis:
addresses three critical aspects of the software supply chain to enhance security: transparency, trust, and automation. First, it investigates transparency as a mechanism to empower developers with accurate and complete insights into the software components integrated into their applications. To this end, the thesis introduces SUNSET and PIP-SBOM, leveraging modeling and SBOMs (Software Bill of Materials) as foundational tools for transparency and security. Second, it examines software trust, focusing on the effectiveness of reproducible builds in major ecosystems and proposing solutions to bolster their adoption. Finally, it emphasizes the role of automation in modern software management, particularly in ensuring user safety and application reliability. This includes developing a tool for automated security testing of GitHub Actions and analyzing the permission models of prominent platforms like GitHub, GitLab, and BitBucket.
Debian developer Simon Josefsson published two reproducibility-related blog posts this month. The first was on the topic of Reproducible Software Releases which discusses some techniques and gotchas that can be encountered when generating reproducible source packages ie. ensuring that the source code archives that open-source software projects release can be reproduced by others. Simon s second post builds on his earlier experiments with reproducing parts of Trisquel/Debian. Titled On Binary Distribution Rebuilds, it discusses potential methods to bootstrap a binary distribution like Debian from some other bootstrappable environment like Guix.
Jochen Sprickerhof uploaded sbuild version 0.88.5 with a change relevant to reproducible builds: specifically, the build_as_root_when_needed functionality still supports older versions of dpkg(1). []
The IzzyOnDroid Android APK repository reached another milestone in March, crossing the 40% coverage mark specifically, more than 42% of the apps in the repository is now reproducible
Thanks to funding by NLnet/Mobifree, the project was also to put more
time into their tooling. For instance, developers can now run easily their own verification builder in less than 5 minutes . This currently supports Debian-based systems, but support for RPM-based systems is incoming. Future work in the pipeline, including documentation, guidelines and helpers for debugging.
Fedora developer Zbigniew J drzejewski-Szmek announced a work-in-progress script called fedora-repro-build which attempts to reproduce an existing package within a Koji build environment. Although the project s README file lists a number of fields will always or almost always vary (and there are a non-zero list of other known issues), this is an excellent first step towards full Fedora reproducibility (see above for more information).
Lastly, in openSUSE news, Bernhard M. Wiedemann posted another monthly update for his work there.
[What] would it take to compromise an entire Linux distribution directly through their public infrastructure? Is it possible to perform such a compromise as simple security researchers with no available resources but time?
diffoscope & strip-nondeterminismdiffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made the following changes, including preparing and uploading versions 290, 291, 292 and 293 and 293 to Debian:
Bug fixes:
file(1) version 5.46 now returns XHTML document for .xhtml files such as those found nested within our .epub tests. []
Also consider .aar files as APK files, at least for the sake of diffoscope. []
Require the new, upcoming, version of file(1) and update our quine-related testcase. []
Codebase improvements:
Ensure all calls to our_check_output in the ELF comparator have the potential CalledProcessError exception caught. [][]
Correct an import masking issue. []
Add a missing subprocess import. []
Reformat openssl.py. []
Update copyright years. [][][]
In addition, Ivan Trubach contributed a change to ignore the st_size metadata entry for directories as it is essentially arbitrary and introduces unnecessary or even spurious changes. []
Website updates
Once again, there were a number of improvements made to our website this month, including:
Herv Boutemy updated the JVM documentation to clarify that the target is rebuild attestation. []
Lastly, Holger Levsen added Julien Malka and Zbigniew J drzejewski-Szmek to our Involved people [][] as well as replaced suggestions to follow us on Twitter/X to follow us on Mastodon instead [][].
Reproducibility testing framework
The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In March, a number of changes were made by Holger Levsen, including:
And finally, node maintenance was performed by Holger Levsen [][][] and Mattia Rizzolo [][].
Upstream patches
The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:
Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:
Since my motivation boost in the beginning of the month caused me
to wrap up a new release of
liboggz, I have used the
same boost to wrap up new editions of
libfishsound,
liboggplay
and
libkate
too. These have been tagged in upstream git, but not yet published on
the Xiph download location. I am waiting for someone with access to
have time to move the tarballs there, I hope it will happen in a few
days. The same is the case for a minor update of liboggz too.
As I was looking at Xiph packages lacking updates, it occurred to
me that there are packages in Debian that have not received a new
upload in a long time. Looking for a way to identify them, I came
across the ltnu script from the
devscripts
package. It can sort by last update, packages maintained by a single
user/group, and is useful to figure out which packages a single
maintainer should have a look at. But I wanted a archive wide
summary. Based on the UDD SQL
query used by ltnu, I ended up with the following command:
#!/bin/sh
env PGPASSWORD=udd-mirror psql --host=udd-mirror.debian.net --user=udd-mirror udd --command="
select source,
max(version) as ver,
max(date) as uploaded
from upload_history
where distribution='unstable' and
source in (select source
from sources
where release='sid')
group by source
order by max(date) asc
limit 50;"
This will sort all source packages in Debian by upload date, and
list the 50 oldest ones. The end result is a list of packages I
suspect could use some attention:
So there are 8 packages last uploaded to unstable in 2011, 12
packages in 2012 and 26 packages in 2013. I suspect their maintainers
need help and we should all offer our assistance. I already contacted
two of them and hope the rest of the Debian community will chip in to
help too. We should ensure any Debian specific patches are passed
upstream if they still exist, that the package is brought up to speed
with the latest Debian policy, as well as ensure the source can built
with the current compiler set in Debian.
As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.
While most podcasts are available on multiple platforms and either offer an
RSS feed or have one that can be
discovered, some are only available in
the form of a YouTube channel. Thankfully, it's possible to both monitor
them for new episodes (i.e. new videos), and time-shift the audio for
later offline listening.
Subscribing to a channel via RSS is possible thanks to the built-in, but not
easily discoverable, RSS feeds. See these
instructions
for how to do it. As an example, the RSS feed for the official Government of
BC channel is
https://www.youtube.com/feeds/videos.xml?channel_id=UC6n9tFQOVepHP3TIeYXnhSA.
When it comes to downloading the audio, the most reliable tool I have found is
yt-dlp. Since the exact arguments needed
to download just the audio as an MP3 are a bit of a mouthful, I wrote a wrapper
script
which also does a few extra things:
cleans up the filename so that it can be stored on any filesystem
adds ID3 tags so that MP3 players can have the metadata they need to
display and group related podcast episodes together
A new version of the RcppAPT
package arrived on CRAN earlier
today. RcppAPT
connects R to the C++ library behind the awesome apt,
apt-get, apt-cache, commands (and their
cache) which powering Debian, Ubuntu and other derivative
distributions.
RcppAPT
allows you to query the (Debian or Ubuntu) package dependency graph at
will, with build-dependencies (if you have deb-src
entries), reverse dependencies, and all other goodies. See the vignette
and examples for illustrations.
This release moves the C++ compilation standard from C++11 to C++17.
I had removed the setting for C++11 last year as compilation by
compiler default worked well enough. But the version at CRAN still carried, which started
to lead to build failures on Debian unstable so it was time for an
update. And rather than implicitly relying on C++17 as selected
by the last two R releases, we made it explicit. Otherwise a few of the
regular package and repository updates have been made, but no new code
or features were added The NEWS entries follow.
Changes in version 0.0.10
(2024-11-29)
Package maintenance updating continuous integration script
versions as well as coverage link from README, and switching to
Authors@R
C++ compilation standards updated to C++17 to comply with
libapt-pkg
FTP master
This month I accepted 270 and rejected 23 packages. The overall number of packages that got accepted was 279.
Debian LTS
This was my hundred-twentieth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.
During my allocated time I uploaded or worked on:
[DLA 3826-1] cups security update for one CVE to prevent arbitrary chmod of files
[#1073519] bullseye-pu: cups/2.3.3op2-3+deb11u7 to fix one CVE
[#1073518] bookworm-pu: cups/2.4.2-3+deb12u6 to fix one CVE
This month handling of the CVE of cups was a bit messy. After lifting the embargo of the CVE, a published patch did not work with all possible combinations of the configuration. In other words, in cases of having only one local domain socket configured, the cupsd did not start and failed with a strange error. Anyway, upstream published a new set of patches, which made cups work again. Unfortunately this happended just before the latest point release for Bullseye and Bookworm, so that the new packages did not make it into the release, but stopped in the corresponding p-u-queues: stable-p-u and old-p-u.
I also continued to work on tiff and last but not least did a week of FD and attended the monthly LTS/ELTS meeting.
Debian ELTS
This month was the seventy-first ELTS month. During my allocated time I tried to upload a new version of cups for Jessie and Stretch. Unfortunately this was stopped due to an autopkgtest error, which I could not reproduce yet.
I also wanted to finally upload a fixed version of exim4. Unfortunately this was stopped due to lots of CI-jobs for Buster. Updates for Buster are now also availble from ELTS, so some stuff had to prepared before the actual switch end of June. Additionally everything was delayed due to a crash of the CI worker. All in all this month was rather ill-fated. At least the exim4 upload will happen/already happened in July.
I also continued to work on an update for libvirt, did a week of FD and attended the LTS/ELTS meeting.
Debian Printing
This month I uploaded new upstream or bugfix versions of:
Welcome to the December 2023 report from the Reproducible Builds project! In these reports we outline the most important things that we have been up to over the past month. As a rather rapid recap, whilst anyone may inspect the source code of free software for malicious flaws, almost all software is distributed to end users as pre-compiled binaries (more).
Reproducibility to affect package migration policy in Debian
In a post summarising the activities of the Debian Release Team at a recent in-person Debian event in Cambridge, UK, Paul Gevers announced a change to the way packages are migrated into the staging area for the next stable Debian release based on its reproducibility status:
The folks from the Reproducibility Project have come a long way since they started working on it 10 years ago, and we believe it s time for the next step in Debian. Several weeks ago, we enabled a migration policy in our migration software that checks for regression in reproducibility. At this moment, that is presented as just for info, but we intend to change that to delays in the not so distant future. We eventually want all packages to be reproducible. To stimulate maintainers to make their packages reproducible now, we ll soon start to apply a bounty [speedup] for reproducible builds, like we ve done with passing autopkgtests for years. We ll reduce the bounty for successful autopkgtests at that moment in time.
Speranza: Usable, privacy-friendly software signing
Kelsey Merrill, Karen Sollins, Santiago Torres-Arias and Zachary Newman have developed a new system called Speranza, which is aimed at reassuring software consumers that the product they are getting has not been tampered with and is coming directly from a source they trust. A write-up on TechXplore.com goes into some more details:
What we have done, explains Sollins, is to develop, prove correct, and demonstrate the viability of an approach that allows the [software] maintainers to remain anonymous. Preserving anonymity is obviously important, given that almost everyone software developers included value their confidentiality. This new approach, Sollins adds, simultaneously allows [software] users to have confidence that the maintainers are, in fact, legitimate maintainers and, furthermore, that the code being downloaded is, in fact, the correct code of that maintainer. []
Nondeterministic Git bundles
Paul Baecher published an interesting blog post on Reproducible git bundles. For those who are not familiar with them, Git bundles are used for the offline transfer of Git objects without an active server sitting on the other side of a network connection. Anyway, Paul wrote about writing a backup system for his entire system, but:
I noticed that a small but fixed subset of [Git] repositories are getting backed up despite having no changes made. That is odd because I would think that repeated bundling of the same repository state should create the exact same bundle. However [it] turns out that for some, repositories bundling is nondeterministic.
Paul goes on to to describe his solution, which involves forcing git to be single threaded makes the output deterministic . The article was also discussed on Hacker News.
Rework the generate-id() function to return deterministic values. We use
a simple incrementing counter and store ids in the 'psvi' member of
nodes which was freed up by previous commits. The presence of an id is
indicated by a new "source node" flag.
This fixes long-standing problems with reproducible builds, see
https://bugzilla.gnome.org/show_bug.cgi?id=751621
This also hardens security, as the old implementation leaked the
difference between a heap and a global pointer, see
https://bugs.chromium.org/p/chromium/issues/detail?id=1356211
The old implementation could also generate the same id for dynamically
created nodes which happened to reuse the same memory. Ids for namespace
nodes were completely broken. They now use the id of the parent element
together with the hex-encoded namespace prefix.
Community updates
There were made a number of improvements to our website, including Chris Lamb fixing the generate-draft script to not blow up if the input files have been corrupted today or even in the past [], Holger Levsen updated the Hamburg 2023 summit to add a link to farewell post [] & to add a picture of a Post-It note. [], and Pol Dellaiera updated the paragraph about tar and the --clamp-mtime flag [].
On our mailing list this month, Bernhard M. Wiedemann posted an interesting summary on some of the reasons why packages are still not reproducible in 2023.
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made a number of changes, including processing objdump symbol comment filter inputs as Python byte (and not str) instances [] and Vagrant Cascadian extended diffoscope support for GNU Guix [] and updated the version in that distribution to version 253 [].
Challenges of Producing Software Bill Of Materials for Java
Musard Balliu, Benoit Baudry, Sofia Bobadilla, Mathias Ekstedt, Martin Monperrus, Javier Ron, Aman Sharma, Gabriel Skoglund, C sar Soto-Valero and Martin Wittlinger (!) of the KTH Royal Institute of Technology in Sweden, have published an article in which they:
deep-dive into 6 tools and the accuracy of the SBOMs they produce for complex open-source Java projects. Our novel insights reveal some hard challenges regarding the accurate production and usage of software bills of materials.
Debian Non-Maintainer campaign
As mentioned in previousreports, the Reproducible Builds team within Debian has been organising a series of online and offline sprints in order to clear the huge backlog of reproducible builds patches submitted by performing so-called NMUs (Non-Maintainer Uploads).
During December, Vagrant Cascadian performed a number of such uploads, including:
In addition, Holger Levsen performed three no-source-change NMUs in order to address the last packages without .buildinfo files in Debian trixie, specifically lorene (0.0.0~cvs20161116+dfsg-1.1), maria (1.3.5-4.2) and ruby-rinku (1.7.3-2.1).
Reproducibility testing framework
The Reproducible Builds project operates a comprehensive testing framework (available at tests.reproducible-builds.org) in order to check packages and other artifacts for reproducibility. In December, a number of changes were made by Holger Levsen:
Detect failures due to rename of the vmlinuz file. []
Improve the grammar of an error message. []
Document that freebsd-jenkins.debian.net has been updated to FreeBSD 14.0. []
In addition, node maintenance was performed by Holger Levsen [] and Vagrant Cascadian [].
Upstream patches
The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:
dkg's New OpenPGP certificate in December 2023
In December of 2023, I'm moving to a new OpenPGP certificate.
You might know my old OpenPGP certificate, which had an fingerprint of
C29F8A0C01F35E34D816AA5CE092EB3A5CA10DBA.
My new OpenPGP certificate has a fingerprint of:
D477040C70C2156A5C298549BB7E9101495E6BF7.
Both certificates have the same set of User IDs:
-----BEGIN PGP PUBLIC KEY BLOCK-----xjMEZXEJyxYJKwYBBAHaRw8BAQdA5BpbW0bpl5qCng/RiqwhQINrplDMSS5JsO/YO+5Zi7HCwAsEHxYKAH0FgmVxCcsDCwkHCRC7fpEBSV5r90cUAAAAAAAeACBzYWx0QG5vdGF0aW9ucy5zZXF1b2lhLXBncC5vcmfUAgfN9tyTSxpxhmHA1r63GiI4v6NQmrrWVLOBRJYuhQMVCggCmwECHgEWIQTUdwQMcMIValwphUm7fpEBSV5r9wAAmaEA/3MvYJMxQdLhIG4UDNMVd2bsovwdcTrReJhLYyFulBrwAQD/j/RS+AXQIVtkcO9bl6zZTAO9x6yfkOZbv0g3eNyrAs0QPGRrZ0BkZWJpYW4ub3JnPsLACwQTFgoAfQWCZXEJywMLCQcJELt+kQFJXmv3RxQAAAAAAB4AIHNhbHRAbm90YXRpb25zLnNlcXVvaWEtcGdwLm9yZ4l+Z3i19Uwjw3CfTNFCDjRsoufMoPOM7vM8HoOEdn/vAxUKCAKbAQIeARYhBNR3BAxwwhVqXCmFSbt+kQFJXmv3AAALZQEAhJsgouepQVV98BHUH6SvWvcKrb8dQEZOvHFbZQQPNWgA/A/DHkjYKnUkCg8Zc+FonqOS/35sHhNA8CwqSQFrtN4KzRc8ZGtnQGZpZnRoaG9yc2VtYW4ubmV0PsLACgQTFgoAfQWCZXEJywMLCQcJELt+kQFJXmv3RxQAAAAAAB4AIHNhbHRAbm90YXRpb25zLnNlcXVvaWEtcGdwLm9yZxLvwkgnslsAuo+IoSa9rv8+nXpbBdab2Ft7n4H9S+d/AxUKCAKbAQIeARYhBNR3BAxwwhVqXCmFSbt+kQFJXmv3AAAtFgD4wqcUfQl7nGLQOcAEHhx8V0Bg8v9ov8GsY1ei1BEFwAD/cxmxmDSO0/tA+x4pd5yIvzgfGYHSTxKS0Ww3hzjuZA7NE0RhbmllbCBLYWhuIEdpbGxtb3LCwA4EExYKAIAFgmVxCcsDCwkHCRC7fpEBSV5r90cUAAAAAAAeACBzYWx0QG5vdGF0aW9ucy5zZXF1b2lhLXBncC5vcmd7X4TgiINwnzh4jar0Pf/b5hgxFPngCFxJSmtr/f0YiQMVCggCmQECmwECHgEWIQTUdwQMcMIValwphUm7fpEBSV5r9wAAMuwBAPtMonKbhGOhOy+8miAb/knJ1cIPBjLupJbjM+NUE1WyAQD1nyGW+XwwMrprMwc320mdJH9B0jdokJZBiN7++0NoBM4zBGVxCcsWCSsGAQQB2kcPAQEHQI19uRatkPSFBXh8usgciEDwZxTnnRZYrhIgiFMybBDQwsC/BBgWCgExBYJlcQnLCRC7fpEBSV5r90cUAAAAAAAeACBzYWx0QG5vdGF0aW9ucy5zZXF1b2lhLXBncC5vcmfCopazDnq6hZUsgVyztl5wmDCmxI169YLNu+IpDzJEtQKbAr6gBBkWCgBvBYJlcQnLCRB3LRYeNc1LgUcUAAAAAAAeACBzYWx0QG5vdGF0aW9ucy5zZXF1b2lhLXBncC5vcmcQglI7G7DbL9QmaDkzcEuk3QliM4NmleIRUW7VvIBHMxYhBHS8BMQ9hghL6GcsBnctFh41zUuBAACwfwEAqDULksr8PulKRcIP6N9NI/4KoznyIcuOHi8qGk4qxMkBAIeV20SPEnWSw9MWAb0eKEcfupzr/C+8vDvsRMynCWsDFiEE1HcEDHDCFWpcKYVJu36RAUlea/cAAFD1AP0YsE3Eeig1tkWaeyrvvMf5Kl1tt2LekTNWDnB+FUG9SgD+Ka8vfPR8wuV8D3y5Y9Qq9xGO+QkEBCW0U1qNypg65QHOOARlcQnLEgorBgEEAZdVAQUBAQdAWTLEa0WmnhUmDBdWXX0ZlYAa4g1CK/fXg0NPOQSteA4DAQgHwsAABBgWCgByBYJlcQnLCRC7fpEBSV5r90cUAAAAAAAeACBzYWx0QG5vdGF0aW9ucy5zZXF1b2lhLXBncC5vcmexrMBZe0QdQ+ZJOZxFkAiwCw2I7yTSF2Ox9GVFWKmAmAKbDBYhBNR3BAxwwhVqXCmFSbt+kQFJXmv3AABcJQD/f4ltpSvLBOBEh/C2dIYadgSuqkCqq0B4WOhFRkWJZlcA/AxqLWG4o8UrrmwrmM42FhgxKtEXwCSHE00u8wR4Up8G=9Yc8-----END PGP PUBLIC KEY BLOCK-----
When I have some reasonable number of certifications, i'll update the
certificate associated with my e-mail addresses on
https://keys.openpgp.org, in DANE, and in WKD. Until then, those
lookups should continue to provide the old certificate.
At the moment I am hard at work putting together the final bits for the AppStream 1.0 release (hopefully to be released this month). The new release comes with many new new features, an improved developer API and removal of most deprecated things (so it carefully breaks compatibility with very old data and the previous C API). One of the tasks for the upcoming 1.0 release was #481 asking about a formal way to distinguish Linux phone applications from desktop applications.
AppStream infamously does not support any is-for-phone label for software components, instead the decision whether something is compatible with a device is based the the device s capabilities and the component s requirements. This allows for truly adaptive applications to describe their requirements correctly, and does not lock us into form factors going into the future, as there are many and the feature range between a phone, a tablet and a tiny laptop is quite fluid.
Of course the match to current device capabilities check does not work if you are a website ranking phone compatibility. It also does not really work if you are a developer and want to know which devices your component / application will actually be considered compatible with. One goal for AppStream 1.0 is to have its library provide more complete building blocks to software centers. Instead of just a here s the data, interpret it according to the specification API, libappstream now interprets the specification for the application and provides API to handle most common operations like checking device compatibility. For developers, AppStream also now implements a few virtual chassis configurations , to roughly gauge which configurations a component may be compatible with.
To test the new code, I ran it against the large Debian and Flatpak repositories to check which applications are considered compatible with what chassis/device type already. The result was fairly disastrous, with many applications not specifying compatibility correctly (many do, but it s by far not the norm!). Which brings me to the actual topic of this blog post: Very few seem to really know how to mark an application compatible with certain screen sizes and inputs! This is most certainly a matter of incomplete guides and good templates, so maybe this post can help with that a bit:
The ultimate cheat-sheet to mark your app chassis-type compatible
As a quick reminder, compatibility is indicated using AppStream s relations system: A requires relation indicates that the system will not run at all or will run terribly if the requirement is not met. If the requirement is not met, it should not be installable on a system. A recommends relation means that it would be advantageous to have the recommended items, but it s not essential to run the application (it may run with a degraded experience without the recommended things though). And a supports relation means a given interface/device/control/etc. is supported by this application, but the application may work completely fine without it.
I have a desktop-only application
A desktop-only application is characterized by needing a larger screen to fit the application, and requiring a physical keyboard and accurate mouse input. This type is assumed by default if no capabilities are set for an application, but it s better to be explicit. This is the metadata you need:
With this requires relation, you require a small-desktop sized screen (at least 768 device-independent pixels (dp) on its smallest edge) and require a keyboard and mouse to be present / connectable. Of course, if your application needs more minimum space, adjust the requirement accordingly. Note that if the requirement is not met, your application may not be offered for installation.
Note: Device-independent / logical pixels
One logical pixel (= device independent pixel) roughly corresponds to the visual angle of one pixel on a device with a pixel density of 96 dpi (for historical X11 reasons) and a distance from the observer of about 52 cm, making the physical pixel about 0.26 mm in size. When using logical pixels as unit, they might not always map to exact physical lengths as their exact size is defined by the device providing the display. They do however accurately depict the maximum amount of pixels that can be drawn in the depicted direction on the device s display space. AppStream always uses logical pixels when measuring lengths in pixels.
I have an application that works on mobile and on desktop / an adaptive app
Adaptive applications have fewer hard requirements, but a wide range of support for controls and screen sizes. For example, they support touch input, unlike desktop apps. An example MetaInfo snippet for these kind of apps may look like this:
Unlike the pure desktop application, this adaptive application requires a much smaller lowest display edge length, and also supports touch input, in addition to keyboard and mouse/touchpad precision input.
I have a pure phone/table app
Making an application a pure phone application is tricky: We need to mark it as compatible with phones only, while not completely preventing its installation on non-phone devices (even though its UI is horrible, you may want to test the app, and software centers may allow its installation when requested explicitly even if they don t show it by default). This is how to achieve that result:
We require a phone-sized display minimum edge size (adjust to a value that is fit for your app!), but then also recommend the screen to have a smaller edge size than a larger tablet/laptop, while also recommending touch input and not listing any support for keyboard and mouse.
Please note that this blog post is of course not a comprehensive guide, so if you want to dive deeper into what you can do with requires/recommends/suggests/supports, you may want to have a look at the relations tags described in the AppStream specification.
Validation
It is still easy to make mistakes with the system requirements metadata, which is why AppStream 1.0 will provide more commands to check MetaInfo files for system compatibility. Current pre-1.0 AppStream versions already have an is-satisfied command to check if the application is compatible with the currently running operating system:
:~$ appstreamcli is-satisfied ./org.example.adaptive_app.metainfo.xml
Relation check for: */*/*/org.example.adaptive_app/*
Requirements:
Unable to check display size: Can not read information without GUI toolkit access.
Recommendations:
No recommended items are set for this software.
Supported:
Physical keyboard found.
Pointing device (e.g. a mouse or touchpad) found.
This software supports touch input.
In addition to this command, AppStream 1.0 will introduce a new one as well: check-syscompat. This command will check the component against libappstream s mock system configurations that define a most common (whatever that is at the time) configuration for a respective chassis type.
If you pass the --details flag, you can even get an explanation why the component was considered or not considered for a specific chassis type:
:~$ appstreamcli check-syscompat --details ./org.example.phoneapp.metainfo.xml
Chassis compatibility check for: */*/*/org.example.phoneapp/*
Desktop:
Incompatible
recommends: This software recommends a display with its shortest edge
being << 1280 px in size, but the display of this device has 1280 px.
recommends: This software recommends a touch input device.
Laptop:
Incompatible
recommends: This software recommends a display with its shortest edge
being << 1280 px in size, but the display of this device has 1280 px.
recommends: This software recommends a touch input device.
Server:
Incompatible
requires: This software needs a display for graphical content.
recommends: This software needs a display for graphical content.
recommends: This software recommends a touch input device.
Tablet:
Compatible (100%)
Handset:
Compatible (100%)
I hope this is helpful for people. Happy metadata writing!
The status quo
Back in 2015, I bought an off-the-shelf NAS, a QNAP TS-453mini, to act as my file store and Plex server. I had previously owned a Synology box, and whilst I liked the Synology OS and experience, the hardware was underwhelming. I loaded up the successor QNAP with four 5TB drives in RAID10, and moved all my files over (after some initial DoA drive issues were handled).
QNAP TS-453mini product photo
That thing has been in service for about 8 years now, and it s been a mixed bag. It was definitely more powerful than the predecessor system, but it was clear that QNAP s OS was not up to the same standard as Synology s perhaps best exemplified by HappyGet 2 , the QNAP webapp for downloading videos from streaming services like YouTube, whose icon is a straight rip-off of StarCraft 2. On its own, meaningless but a bad omen for overall software quality
The logo for QNAP HappyGet 2 and Blizzard s StarCraft 2 side by side
Additionally, the embedded Celeron processor in the NAS turned out to be an issue for some cases. It turns out, when playing back videos with subtitles, most Plex clients do not support subtitles properly instead they rely on the Plex server doing JIT transcoding to bake the subtitles directly into the video stream. I discovered this with some Blu-Ray rips of Game of Thrones some episodes would play back fine on my smart TV, but episodes with subtitled Dothraki speech would play at only 2 or 3 frames per second.
The final straw was a ransomware attack, which went through all my data and locked every file below a 60MiB threshold. Practically all my music gone. A substantial collection of downloaded files, all gone. Some of these files had been carried around since my college days digital rarities, or at least digital detritus I felt a real sense of loss at having to replace. This episode was caused by a ransomware targeting specific vulnerabilities in the QNAP OS, not an error on my part.
So, I decided to start planning a replacement with:
A non-garbage OS, whilst still being a NAS-appliance type offering (not an off-the-shelf Linux server distro)
Full remote management capabilities
A small form factor comparable to off-the-shelf NAS
A powerful modern CPU capable of transcoding high resolution video
All flash storage, no spinning rust
At the time, no consumer NAS offered everything (The Asustor FS6712X exists now, but didn t when this project started), so I opted to go for a full DIY rather than an appliance not the first time I ve jumped between appliances and DIY for home storage.
Selecting the core of the system
There aren t many companies which will sell you a small motherboard with IPMI. Supermicro is a bust, so is Tyan. But ASRock Rack, the server division of third-tier motherboard vendor ASRock, delivers. Most of their boards aren t actually compliant Mini-ITX size, they re a proprietary Deep Mini-ITX with the regular screw holes, but 40mm of extra length (and a commensurately small list of compatible cases). But, thankfully, they do have a tiny selection of boards without the extra size, and I stumbled onto the X570D4I-2T, a board with an AMD AM4 socket and the mature X570 chipset. This board can use any AMD Ryzen chip (before the latest-gen Ryzen 7000 series); has built in dual 10 gigabit ethernet; IPMI; four (laptop-sized) RAM slots with full ECC support; one M.2 slot for NVMe SSD storage; a PCIe 16x slot (generally for graphics cards, but we live in a world of possibilities); and up to 8 SATA drives OR a couple more NVMe SSDs. It s astonishingly well featured, just a shame it costs about $450 compared to a good consumer-grade Mini ITX AM4 board costing less than half that.
I was so impressed with the offering, in fact, that I crowed about it on Mastodon and ended up securing ASRock another sale, with someone else looking into a very similar project to mine around the same timespan.
The next question was the CPU. An important feature of a system expected to run 24/7 is low power, and AM4 chips can consume as much as 130W under load, out of the box. At the other end, some models can require as little as 35W under load the OEM-only GE suffix chips, which are readily found for import on eBay. In their PRO variant, they also support ECC (all non-G Ryzen chips support ECC, but only Pro G chips do). The top of the range 8 core Ryzen 7 PRO 5750GE is prohibitively expensive, but the slightly weaker 6 core Ryzen 5 PRO 5650GE was affordable, and one arrived quickly from Hong Kong. Supplemented with a couple of cheap 16 GiB SODIMM sticks of DDR4 PC-3200 direct from Micron for under $50 a piece, that left only cooling as an unsolved problem to get a bootable test system.
The official support list for the X570D4I-2T only includes two rackmount coolers, both expensive and hard to source. The reason for such a small list is the non standard cooling layout of the board instead of an AM4 hole pattern with the standard plastic AM4 retaining clips, it has an Intel 115x hole pattern with a non-standard backplate (Intel 115x boards have no backplate, the stock Intel 115x cooler attaches to the holes with push pins). As such every single cooler compatibility list excludes this motherboard. However, the backplate is only secured with a mild glue with minimal pressure and a plastic prying tool it can be removed, giving compatibility with any 115x cooler (which is basically any CPU cooler for more than a decade). I picked an oversized low profile Thermalright AXP120-X67 hoping that its 120mm fan would cool the nearby MOSFETs and X570 chipset too.
Thermalright AXP120-X67, AMD Ryzen 5 PRO 5650GE, ASRock Rack X570D4I-2T, all assembled and running on a flat surface
Testing up to this point
Using a spare ATX power supply, I had enough of a system built to explore the IPMI and UEFI instances, and run MemTest86 to validate my progress. The memory test ran without a hitch and confirmed the ECC was working, although it also showed that the memory was only running at 2933 MT/s instead of the rated 3200 MT/s (a limit imposed by the motherboard, as higher speeds are considered overclocking). The IPMI interface isn t the best I ve ever used by a long shot, but it s minimum viable and allowed me to configure the basics and boot from media entirely via a Web browser.
Memtest86 showing test progress, taken from IPMI remote control window
One sad discovery, however, which I ve never seen documented before, on PCIe bifurcation.
With PCI Express, you have a number of lanes which are allocated in groups by the motherboard and CPU manufacturer. For Ryzen prior to Ryzen 7000, that s 16 lanes in one slot for the graphics card; 4 lanes in one M.2 connector for an SSD; then 4 lanes connecting the CPU to the chipset, which can offer whatever it likes for peripherals or extra lanes (bottlenecked by that shared 4x link to the CPU, if it comes down to it).
It s possible, with motherboard and CPU support, to split PCIe groups up for example an 8x slot could be split into two 4x slots (eg allowing two NVMe drives in an adapter card NVME drives these days all use 4x). However with a Cezanne Ryzen with integrated graphics, the 16x graphics card slot cannot be split into four 4x slots (ie used for for NVMe drives) the most bifurcation it allows is 8x4x4x, which is useless in a NAS.
Screenshot of PCIe 16x slot bifurcation options in UEFI settings, taken from IPMI remote control window
As such, I had to abandon any ideas of an all-NVMe NAS I was considering: the 16x slot split into four 4x, combined with two 4x connectors fed by the X570 chipset, to a total of 6 NVMe drives. 7.6TB U.2 enterprise disks are remarkably affordable (cheaper than consumer SATA 8TB drives), but alas, I was locked out by my 5650GE. Thankfully I found out before spending hundreds on a U.2 hot swap bay. The NVMe setup would be nearly 10x as fast as SATA SSDs, but at least the SATA SSD route would still outperform any spinning rust choice on the market (including the fastest 10K RPM SAS drives)
Containing the core
The next step was to pick a case and power supply. A lot of NAS cases require an SFX (rather than ATX) size supply, so I ordered a modular SX500 unit from Silverstone. Even if I ended up with a case requiring ATX, it s easy to turn an SFX power supply into ATX, and the worst result is you have less space taken up in your case, hardly the worst problem to have.
That said, on to picking a case. There s only one brand with any cachet making ITX NAS cases, Silverstone. They have three choices in an appropriate size: CS01-HS, CS280, and DS380. The problem is, these cases are all badly designed garbage. Take the CS280 as an example, the case with the most space for a CPU cooler. Here s how close together the hotswap bay (right) and power supply (left) are:
Internal image of Silverstone CS280 NAS build. Image stolen from ServeTheHome
With actual cables connected, the cable clearance problem is even worse:
Internal image of Silverstone CS280 NAS build. Image stolen from ServeTheHome
Remember, this is the best of the three cases for internal layout, the one with the least restriction on CPU cooler height. And it s garbage! Total hot garbage! I decided therefore to completely skip the NAS case market, and instead purchase a 5.25 -to-2.5 hot swap bay adapter from Icy Dock, and put it in an ITX gamer case with a 5.25 bay. This is no longer a served market 5.25 bays are extinct since nobody uses CD/DVD drives anymore. The ones on the market are really new old stock from 2014-2017: The Fractal Design Core 500, Cooler Master Elite 130, and Silverstone SUGO 14. Of the three, the Fractal is the best rated so I opted to get that one however it seems the global supply of new old stock fully dried up in the two weeks between me making a decision and placing an order leaving only the Silverstone case.
Icy Dock have a selection of 8-bay 2.5 SATA 5.25 hot swap chassis choices in their ToughArmor MB998 series. I opted for the ToughArmor MB998IP-B, to reduce cable clutter it requires only two SFF-8611-to-SF-8643 cables from the motherboard to serve all eight bays, which should make airflow less of a mess. The X570D4I-2T doesn t have any SATA ports on board, instead it has two SFF-8611 OCuLink ports, each supporting 4 PCI Express lanes OR 4 SATA connectors via a breakout cable. I had hoped to get the ToughArmor MB118VP-B and run six U.2 drives, but as I said, the PCIe bifurcation issue with Ryzen G chips meant I wouldn t be able to run all six bays successfully.
NAS build in Silverstone SUGO 14, mid build, panels removedSilverstone SUGO 14 from the front, with hot swap bay installed
Actual storage for the storage server
My concept for the system always involved a fast boot/cache drive in the motherboard s M.2 slot, non-redundant (just backups of the config if the worst were to happen) and separate storage drives somewhere between 3.8 and 8 TB each (somewhere from $200-$350). As a boot drive, I selected the Intel Optane SSD P1600X 58G, available for under $35 and rated for 228 years between failures (or 11,000 complete drive rewrite cycles).
So, on to the big expensive choice: storage drives. I narrowed it down to two contenders: new-old-stock Intel D3-S4510 3.84TB enterprise drives, at about $200, or Samsung 870 QVO 8TB consumer drives, at about $375. I did spend a long time agonizing over the specification differences, the ZFS usage reports, the expected lifetime endurance figures, but in reality, it came down to price $1600 of expensive drives vs $3200 of even more expensive drives. That s 27TB of usable capacity in RAID-Z1, or 23TB in RAID-Z2. For comparison, I m using about 5TB of the old NAS, so that s a LOT of overhead for expansion.
Storage SSD loaded into hot swap sled
Booting up
Bringing it all together is the OS. I wanted an appliance NAS OS rather than self-administering a Linux distribution, and after looking into the surrounding ecosystems, decided on TrueNAS Scale (the beta of the 2023 release, based on Debian 12).
TrueNAS Dashboard screenshot in browser window
I set up RAID-Z1, and with zero tuning (other than enabling auto-TRIM), got the following performance numbers:
Finally, the numbers reported on the old NAS with four 7200 RPM hard disks in RAID 10:
IOPS
Bandwidth
4k random writes
430
1.7 MiB/s
4k random reads
8006
32 MiB/s
Sequential writes
311 MiB/s
Sequential reads
566 MiB/s
Performance seems pretty OK. There s always going to be an overhead to RAID. I ll settle for the 45x improvement on random writes vs. its predecessor, and 4.5x improvement on random reads. The sequential write numbers are gonna be impacted by the size of the ZFS cache (50% of RAM, so 16 GiB), but the rest should be a reasonable indication of true performance.
It took me a little while to fully understand the TrueNAS permissions model, but I finally got Plex configured to access data from the same place as my SMB shares, which have anonymous read-only access or authenticated write access for myself and my wife, working fine via both Linux and Windows.
And that s it! I built a NAS. I intend to add some fans and more RAM, but that s the build. Total spent: about $3000, which sounds like an unreasonable amount, but it s actually less than a comparable Synology DiskStation DS1823xs+ which has 4 cores instead of 6, first-generation AMD Zen instead of Zen 3, 8 GiB RAM instead of 32 GiB, no hardware-accelerated video transcoding, etc. And it would have been a whole lot less fun!
The final system, powered up
(Also posted on PCPartPicker)
Welcome to the June 2023 report from the Reproducible Builds project
In our reports, we outline the most important things that we have been up to over the past month. As always, if you are interested in contributing to the project, please visit our Contribute page on our website.
We are very happy to announce the upcoming Reproducible Builds Summit which set to take place from October 31st November 2nd 2023, in the vibrant city of Hamburg, Germany.
Our summits are a unique gathering that brings together attendees from diverse projects, united by a shared vision of advancing the Reproducible Builds effort. During this enriching event, participants will have the opportunity to engage in discussions, establish connections and exchange ideas to drive progress in this vital field. Our aim is to create an inclusive space that fosters collaboration, innovation and problem-solving. We are thrilled to host the seventh edition of this exciting event, following the success of previous summits in various iconic locations around the world, including Venice, Marrakesh, Paris, Berlin and Athens.
If you re interesting in joining us this year, please make sure to read the event page] which has more details about the event and location. (You may also be interested in attending PackagingCon 2023 held a few days before in Berlin.)
This month, Vagrant Cascadian will present at FOSSY 2023 on the topic of Breaking the Chains of Trusting Trust:
Corrupted build environments can deliver compromised cryptographically signed binaries. Several exploits in critical supply chains have been demonstrated in recent years, proving that this is not just theoretical. The most well secured build environments are still single points of failure when they fail. [ ] This talk will focus on the state of the art from several angles in related Free and Open Source Software projects, what works, current challenges and future plans for building trustworthy toolchains you do not need to trust.
Hosted by the Software Freedom Conservancy and taking place in Portland, Oregon, FOSSY aims to be a community-focused event: Whether you are a long time contributing member of a free software project, a recent graduate of a coding bootcamp or university, or just have an interest in the possibilities that free and open source software bring, FOSSY will have something for you . More information on the event is available on the FOSSY 2023 website, including the full programme schedule.
Marcel Fourn , Dominik Wermke, William Enck, Sascha Fahl and Yasemin Acar recently published an academic paper in the 44th IEEE Symposium on Security and Privacy titled It s like flossing your teeth: On the Importance and Challenges of Reproducible Builds for Software Supply Chain Security . The abstract reads as follows:
The 2020 Solarwinds attack was a tipping point that caused a heightened awareness about the security of the software supply chain and in particular the large amount of trust placed in build systems. Reproducible Builds (R-Bs) provide a strong foundation to build defenses for arbitrary attacks against build systems by ensuring that given the same source code, build environment, and build instructions, bitwise-identical artifacts are created.
However, in contrast to other papers that touch on some theoretical aspect of reproducible builds, the authors paper takes a different approach. Starting with the observation that much of the software industry believes R-Bs are too far out of reach for most projects and conjoining that with a goal of to help identify a path for R-Bs to become a commonplace property , the paper has a different methodology:
We conducted a series of 24 semi-structured expert interviews with participants from the Reproducible-Builds.org project, and iterated on our questions with the reproducible builds community. We identified a range of motivations that can encourage open source developers to strive for R-Bs, including indicators of quality, security benefits, and more efficient caching of artifacts. We identify experiences that help and hinder adoption, which heavily include communication with upstream projects. We conclude with recommendations on how to better integrate R-Bs with the efforts of the open source and free software community.
Vagrant Cascadian mentioned that Packaging Con 2023 is being held in Berlin, the weekend before the Reproducible Builds summit later this year. In particular, Vagrant noticed that the Call for Proposals (CFP) closes at the end of July.
Larry Doolittle was searching Usenet archives and discovered a thread from December 1999 titled Time independent checksum(cksum) on comp.unix.programming. Larry notes that it starts with Jayan asking about comparing binaries that might have difference in their embedded timestamps (that is, perhaps, Foreshadowing diffoscope, amiright? ) and goes on to observe that:
The antagonist is David Schwartz, who correctly says There are dozens of complex reasons why what seems to be the same sequence of operations might produce different end results, but goes on to say I totally disagree with your general viewpoint that compilers must provide for reproducability [sic].
Dwight Tovey and I (Larry Doolittle) argue for reproducible builds. I assert Any program especially a mission-critical program like a compiler that cannot reproduce a result at will is broken. Also it s commonplace to take a binary from the net, and check to see if it was trojaned by attempting to recreate it from source.
Distribution work
27 reviews of Debian packages were added, 40 were updated and 8 were removed this month adding to our knowledge about identified issues. A new randomness_in_documentation_generated_by_mkdocs toolchain issue was added by Chris Lamb [], and the deterministic flag on the paths_vary_due_to_usrmerge issue as we are not currently testing usrmerge issues [] issues.
Roland Clobus posted his 18th update of the status of reproducible Debian ISO images on our mailing list. Roland reported that all major desktops build reproducibly with bullseye, bookworm, trixie and sid , but he also mentioned amongst many changes that not only are the non-free images being built (and are reproducible) but that the live images are generated officially by Debian itself. []
Jan-Benedict Glaw noticed a problem when building NetBSD for the VAX architecture. Noting that Reproducible builds [are] probably not as reproducible as we thought , Jan-Benedict goes on to describe that when two builds from different source directories won t produce the same result and adds various notes about sub-optimal handling of the CFLAGS environment variable. []
F-Droid added 21 new reproducible apps in June, resulting in a new record of 145 reproducible apps in total. []. (This page now sports missing data for March May 2023.) F-Droid contributors also reported an issue with broken resources in APKs making some builds unreproducible. []
Bernhard M. Wiedemann published another monthly report about reproducibility within openSUSE
Testing framework
The Reproducible Builds project operates a comprehensive testing framework (available at tests.reproducible-builds.org) in order to check packages and other artifacts for reproducibility. In June, a number of changes were made by Holger Levsen, including:
Additions to a (relatively) new Documented Jenkins Maintenance (djm) script to automatically shrink a cache & save a backup of old data [], automatically split out previous months data from logfiles into specially-named files [], prevent concurrent remote logfile fetches by using a lock file [] and to add/remove various debugging statements [].
Updates to the automated system health checks to, for example, to correctly detect new kernel warnings due to a wording change [] and to explicitly observe which old/unused kernels should be removed []. This was related to an improvement so that various kernel issues on Ubuntu-based nodes are automatically fixed. []
Holger and Vagrant Cascadian updated all thirty-five hosts running Debian on the amd64, armhf, and i386 architectures to Debian bookworm, with the exception of the Jenkins host itself which will be upgraded after the release of Debian 12.1. In addition, Mattia Rizzolo updated the email configuration for the @reproducible-builds.org domain to correctly accept incoming mails from jenkins.debian.net [] as well as to set up DomainKeys Identified Mail (DKIM) signing []. And working together with Holger, Mattia also updated the Jenkins configuration to start testing Debian trixie which resulted in stopped testing Debian buster. And, finally, Jan-Benedict Glaw contributed patches for improved NetBSD testing.
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:
While visiting a convention during Easter, it occurred to me that
it would be great if I could have a digital Dictaphone with
transcribing capabilities, providing me with texts to cut-n-paste into
stuff I need to write. The background is that long drives often bring
up the urge to write on texts I am working on, which of course is out
of the question while driving. With the release of
OpenAI Whisper, this
seem to be within reach with Free Software, so I decided to give it a
go. OpenAI Whisper is a Linux based neural network system to read in
audio files and provide text representation of the speech in that
audio recording. It handle multiple languages and according to its
creators even can translate into a different language than the spoken
one. I have not tested the latter feature. It can either use the CPU
or a GPU with CUDA support. As far as I can tell, CUDA in practice
limit that feature to NVidia graphics cards. I have few of those, as
they do not work great with free software drivers, and have not tested
the GPU option. While looking into the matter, I did discover some
work to provide CUDA support on non-NVidia GPUs, and some work with
the library used by Whisper to port it to other GPUs, but have not
spent much time looking into GPU support yet. I've so far used an old
X220 laptop as my test machine, and only transcribed using its
CPU.
As it from a privacy standpoint is unthinkable to use computers
under control of someone else (aka a "cloud" service) to transcribe
ones thoughts and personal notes, I want to run the transcribing
system locally on my own computers. The only sensible approach to me
is to make the effort I put into this available for any Linux user and
to upload the needed packages into Debian. Looking at Debian Bookworm, I
discovered that only three packages were missing,
tiktoken,
triton, and
openai-whisper. For a while
I also believed
ffmpeg-python was
needed, but as its
upstream
seem to have vanished I found it safer
to rewrite
whisper to stop depending on in than to introduce ffmpeg-python
into Debian. I decided to place these packages under the umbrella of
the Debian Deep
Learning Team, which seem like the best team to look after such
packages. Discussing the topic within the group also made me aware
that the triton package was already a future dependency of newer
versions of the torch package being planned, and would be needed after
Bookworm is released.
All required code packages have been now waiting in
the Debian NEW
queue since Wednesday, heading for Debian Experimental until
Bookworm is released. An unsolved issue is how to handle the neural
network models used by Whisper. The default behaviour of Whisper is
to require Internet connectivity and download the model requested to
~/.cache/whisper/ on first invocation. This obviously would
fail the
deserted island test of free software as the Debian packages would
be unusable for someone stranded with only the Debian archive and solar
powered computer on a deserted island.
Because of this, I would love to include the models in the Debian
mirror system. This is problematic, as the models are very large
files, which would put a heavy strain on the Debian mirror
infrastructure around the globe. The strain would be even higher if
the models change often, which luckily as far as I can tell they do
not. The small model, which according to its creator is most useful
for English and in my experience is not doing a great job there
either, is 462 MiB (deb is 414 MiB). The medium model, which to me
seem to handle English speech fairly well is 1.5 GiB (deb is 1.3 GiB)
and the large model is 2.9 GiB (deb is 2.6 GiB). I would assume
everyone with enough resources would prefer to use the large model for
highest quality. I believe the models themselves would have to go
into the non-free part of the Debian archive, as they are not really
including any useful source code for updating the models. The
"source", aka the model training set, according to the creators
consist of "680,000 hours of multilingual and multitask supervised
data collected from the web", which to me reads material with both
unknown copyright terms, unavailable to the general public. In other
words, the source is not available according to the Debian Free
Software Guidelines and the model should be considered non-free.
I asked the Debian FTP masters for advice regarding uploading a
model package on their IRC channel, and based on the feedback there it
is still unclear to me if such package would be accepted into the
archive. In any case I wrote build rules for a
OpenAI
Whisper model package and
modified the
Whisper code base to prefer shared files under /usr/ and
/var/ over user specific files in ~/.cache/whisper/
to be able to use these model packages, to prepare for such
possibility. One solution might be to include only one of the models
(small or medium, I guess) in the Debian archive, and ask people to
download the others from the Internet. Not quite sure what to do
here, and advice is most welcome (use the debian-ai mailing list).
To make it easier to test the new packages while I wait for them to
clear the NEW queue, I created an APT source targeting bookworm. I
selected Bookworm instead of Bullseye, even though I know the latter
would reach more users, is that some of the required dependencies are
missing from Bullseye and I during this phase of testing did not want
to backport a lot of packages just to get up and running.
Here is a recipe to run as user root if you want to test OpenAI
Whisper using Debian packages on your Debian Bookworm installation,
first adding the APT repository GPG key to the list of trusted keys,
then setting up the APT repository and finally installing the packages
and one of the models:
curl https://geekbay.nuug.no/~pere/openai-whisper/D78F5C4796F353D211B119E28200D9B589641240.asc \
-o /etc/apt/trusted.gpg.d/pere-whisper.asc
mkdir -p /etc/apt/sources.list.d
cat > /etc/apt/sources.list.d/pere-whisper.list <<EOF
deb https://geekbay.nuug.no/~pere/openai-whisper/ bookworm main
deb-src https://geekbay.nuug.no/~pere/openai-whisper/ bookworm main
EOF
apt update
apt install openai-whisper
The package work for me, but have not yet been tested on any other
computer than my own. With it, I have been able to (badly) transcribe
a 2 minute 40 second Norwegian audio clip to test using the small
model. This took 11 minutes and around 2.2 GiB of RAM. Transcribing
the same file with the medium model gave a accurate text in 77 minutes
using around 5.2 GiB of RAM. My test machine had too little memory to
test the large model, which I believe require 11 GiB of RAM. In
short, this now work for me using Debian packages, and I hope it will
for you and everyone else once the packages enter Debian.
Now I can start on the audio recording part of this project.
As usual, if you use Bitcoin and want to show your support of my
activities, please send Bitcoin donations to my address
15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.
Here s my (forty-first) monthly but brief update about the activities I ve done in the F/L/OSS world.
Debian
This was my 50th month of actively contributing to Debian.
I became a DM in late March 2019 and a DD on Christmas 19! \o/
There s a bunch of things I do, both, technical and non-technical. Here are the things I did this month:
libyang2 (2.1.30-2) - Adding DEP8 test for yangre.
redmine (5.0.4-3) - Add patch to stop unnecessary recursive chown ing (Fixes: #1022816, #1022817).
redmine (5.0.4-4) - Set DH_RUBY_IGNORE_TESTS to all (Fixes: #1031308).
python-jira (3.4.1-1) - New upstream version, v3.4.1.
Others
Looked up some Release team documentation.
Sponsored php-font-lib and php-dompdf-svg-lib for William.
Granted DM rights for php-dompdf.
Mentoring for newcomers.
Reviewed micro bits for Nilesh, new uploads and changes.
Ruby sprints.
Bug work (on BTS and #debian-ruby) for rails and redmine.
Moderation of -project mailing list.
A huge thanks to Freexian for sponsoring my Debian work and Entrouvert for sponsoring the Redmine backports. :D
Ubuntu
This was my 25th month of actively contributing to Ubuntu.
Now that I joined Canonical to work on Ubuntu full-time, there s a bunch of things I do! \o/
I mostly worked on different things, I guess.
I was too lazy to maintain a list of things I worked on so there s
no concrete list atm. Maybe I ll get back to this section later or
will start to list stuff from the fall, as I was doing before. :D
Debian (E)LTS
Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success.
And Debian Extended LTS (ELTS) is its sister project, extending support to the stretch and jessie release (+2 years after LTS support).
This was my forty-first month as a Debian LTS and thirty-second month as a Debian ELTS paid contributor.
I worked for 24.25 hours for LTS and 28.50 hours for ELTS.
LTS CVE Fixes and Announcements:
Fixed CVE-2022-47016 for tmux and uploaded to buster via 2.8-3+deb10u1.
But decided to not roll the DLA for the package as the CVE got rejected upstream.
Worked on ruby-rails-html-sanitize and added notes to the security-tracker.
TL;DR: we need newer methods in ruby-loofah to make the patches for ruby-rails-html-sanitize backportable.
Started to look at other set of packages meanwhile.
ELTS CVE Fixes and Announcements:
Issued ELA 813-1, fixing CVE-2017-12618 and CVE-2022-25147, for apr-util.
For Debian 8 jessie, these problems have been fixed in version 1.5.4-1+deb8u1.
For Debian 9 stretch, these problems have been fixed in version 1.5.4-3+deb9u1.
Issued ELA 815-1, fixing CVE-2022-44792 and CVE-2022-44793, for net-snmp.
For Debian 8 jessie, these problems have been fixed in version 5.7.2.1+dfsg-1+deb8u6.
For Debian 9 stretch, these problems have been fixed in version 5.7.3+dfsg-1.7+deb9u5.
Helped facilitate RabbitMQ s update queries by one of our customers.
Started to look at other set of packages meanwhile.
Sunset, Witch Wells Arizona
Another busy week!
In the snap world, I have been busy trying to solve the problem of core20 snaps needing security updates and focal is no longer supported in KDE Neon. So I have created a ppa at https://launchpad.net/~scarlettmoore/+archive/ubuntu/kf5-5.99-focal-updates/+packages
Which of course presents more work, as kf5 5.99.0 requires qt5 5.15.7. Sooo this is a WIP.
Snapcraft kde-neon-extension is moving along as I learn the python ways of formatting, and fixing some issues in my tests.
In the Debian world, I am sad to report Mycroft-AI has gone bust, however the packaging efforts are not in vain as the project has been forked to https://github.com/orgs/OpenVoiceOS/repositories and should be relatively easy to migrate.
I have spent some time verifying the libappimage in buster is NOT vulnerable with CVE-2020-25265 as the code wasn t introduced yet.
Skanpage, plasma-bigscreen both have source uploads so the can migrate to testing to hopefully make it into bookworm!
As many of you know, I am seeking employment. I am a hard worker, that thrives on learning new things. I am a self starter, knowledge sponge, and eager to be an asset to < insert your company here > ! Meanwhile, as interview processes are much longer than I remember and the industry exploding in layoffs, I am coming up short on living expenses as my unemployment lingers on. Please consider donating to my gofundme. Thank you for your consideration.I still have a ways to go to cover my bills this month, I will continue with my work until I cannot, I hate asking, but please consider a donation. Thank you!GoFundMe
Like each month, have a look at the work funded by 
A new version of the 
Physical keyboard found.
QNAP TS-453mini product photo
The logo for QNAP HappyGet 2 and Blizzard s StarCraft 2 side by side
Thermalright AXP120-X67, AMD Ryzen 5 PRO 5650GE, ASRock Rack X570D4I-2T, all assembled and running on a flat surface
Memtest86 showing test progress, taken from IPMI remote control window
Screenshot of PCIe 16x slot bifurcation options in UEFI settings, taken from IPMI remote control window
Internal image of Silverstone CS280 NAS build. Image stolen from 
Internal image of Silverstone CS280 NAS build. Image stolen from 
NAS build in Silverstone SUGO 14, mid build, panels removed
Silverstone SUGO 14 from the front, with hot swap bay installed
Storage SSD loaded into hot swap sled
TrueNAS Dashboard screenshot in browser window
The final system, powered up
Sunset, Witch Wells Arizona