• Codethink
• Mythic Beasts

Motivation
Andy (RattusRattus) and I have been formalising instructions for using Pete Batard's version of Tianocore (and therefore UEFI booting) for the Raspberry Pi 4 together with a Debian arm64 netinst to make a modified Debian installer on a USB stick which "just works" for a Raspberry Pi 4.
Thanks also to Steve McIntyre for initial notes that got this working for us and also to Emmanuele Rocca for putting up some useful instructions for copying.

Recipe

Plug in a USB stick - use dmesg or your favourite method to see how it is identified.

Make a couple of mount points under /mnt - /mnt/data and /mnt/cdrom

1. Grab a USB stick, Partition using MBR. Make a single VFAT
partition, type 0xEF (i.e. EFI System Partition)

For a USB stick (identified as sdX) below:
$ sudo parted --script /dev/sdX mklabel msdos $ sudo parted --script /dev/sdX mkpart primary fat32 0% 100% $ sudo mkfs.vfat /dev/sdX1 $ sudo mount /dev/sdX1 /mnt/data/

Download an arm64 netinst.iso

https://cdimage.debian.org/debian-cd/current/arm64/iso-cd/debian-12.2.0-arm64-netinst.iso

2. Copy the complete contents of partition *1* from a Debian arm64
installer image into the filesystem (partition 1 is the installer
stuff itself) on the USB stick, in /

$ sudo kpartx -v -a debian-12.2.0-arm64-netinst.iso # Mount the first partition on the ISO and copy its contents to the stick $ sudo mount /dev/mapper/loop0p1 /mnt/cdrom/ $ sudo rsync -av /mnt/cdrom/ /mnt/data/ $ sudo umount /mnt/cdrom

3. Copy the complete contents of partition *2* from that Debian arm64
installer image into that filesystem (partition 2 is the ESP) on
the USB stick, in /

# Same story with the second partition on the ISO

$ sudo mount /dev/mapper/loop0p2 /mnt/cdrom/

$ sudo rsync -av /mnt/cdrom/ /mnt/data/ $ sudo umount /mnt/cdrom

$ sudo kpartx -d debian-testing-amd64-netinst.iso $ sudo umount /mnt/data

4. Grab the rpi edk2 build from https://github.com/pftf/RPi4/releases
(I used 1.35) and extract it. I copied the files there into *2*
places for now on the USB stick:

/ (so the Pi will boot using it)
/rpi4 (so we can find the files again later)

5. Add the preseed.cfg file (attached) into *both* of the two initrd
files on the USB stick

- /install.a64/initrd.gz and
- /install.a64/gtk/initrd.gz

cpio is an awful tool to use :-(. In each case:

$ cp /path/to/initrd.gz .
$ gunzip initrd.gz
$ echo preseed.cfg cpio -H newc -o -A -F initrd

$ gzip -9v initrd

$ cp initrd.gz /path/to/initrd.gz

If you look at the preseed file, it will do a few things:

- Use an early_command to unmount /media (to work around Debian bug
#1051964)

- Register a late_command call for /cdrom/finish-rpi (the next
file - see below) to run at the end of the installation.

- Force grub installation also to the EFI removable media path,
needed as the rpi doesn't store EFI boot variables.

- Stop the installer asking for firmware from removable media (as
the rpi4 will ask for broadcom bluetooth fw that we can't
ship. Can be ignored safely.)

6. Copy the finish-rpi script (attached) into / on the USB stick. It
will be run at the end of the installation, triggered via the
preseed. It does a couple of things:

- Copy the edk2 firmware files into the ESP on the system that's
just been installer

- Remove shim-signed from the installed systems, as there's a bug
that causes it to fail on rpi4. I need to dig into this to see
what the issue is.

That's it! Run the installer as normal, all should Just Work (TM).

BlueTooth didn't quite work : raspberrypi-firmware didn't install until adding a symlink for boot/efi to /boot/firmware

20231127 - This may not be necessary because raspberrypi-firmware path has been fixed

Preseed.cfg
# The preseed file itself causes a problem - the installer medium is
# left mounted on /medis so things break in cdrom-detect. Let's see if
# we can fix that!
d-i preseed/early_command string umount /media true

# Run our command to do rpi setup before reboot
d-i preseed/late_command string /cdrom/finish-rpi

# Force grub installation to the RM path
grub-efi-arm64 grub2/force_efi_extra_removable boolean true

# Don't prompt for missing firmware from removable media,
# e.g. broadcom bluetooth on the rpi.
d-i hw-detect/load_firmware boolean false

Finish.rpi
!/bin/sh

set -x

grep -q -a RPI4 /sys/firmware/acpi/tables/CSRT
if [ $? -ne 0 ]; then
echo "Not running on a Pi 4, exit!"
exit 0
fi

# Copy the rpi4 firmware binaries onto the installed system.
# Assumes the installer media is mounted on /cdrom.
cp -vr /cdrom/rpi4/. /target/boot/efi/.

# shim-signed doesn't seem happy on rpi4, so remove it
mount --bind /sys /target/sys
mount --bind /proc /target/proc
mount --bind /dev /target/dev

in-target apt-get remove --purge --autoremove -y shim-signed

• Collabora
• Codethink
• Andy Simpkins

• Update the SC with the new text, update the website.
• Check/add support for the non-free-firmware section in various places:
• d-i build
• debian-cd
• debmirror (?)
• ftpsync (?)
• Any others?
• Uploads of firmware packages targeting non-free-firmware.
• Extra d-i code to inform users about what firmware blobs have been loaded and the matching non-free-firmware packages. Plus information about the hardware involved. Maybe a new d-i module / udeb for this? Exact details here still TBD. Probably the biggest individual piece of work here.
• Tweaks to add the non-free-firmware section in the apt-setup module if desired/needed.
• An extra boot option (a debconf variable) to disable loading extra firmware automatically, then exposed as an extra option through the isolinux and GRUB menus. d-i "expert mode" can also be used to tweak this behaviour later, except (obviously) for things like audio firmware that must be loaded early if they're going to be useful at all.
• Update the image build scripts to include the n-f-f packages, only build one type of image. I'll do my best to keep config and support around too for images without n-f-f included, to make it easier to still build those for people who still want them.
• Matching updates to docs, website, wiki etc.
• ...

• Debian to vote on its firmware path in Linux Weekly News
• Debian Begins A General Resolution To Decide What To Do With Non-Free Firmware in Phoronix
• Debian set to start vote on including non-free packages in install media in ITWire
• Why Debian Might Include Non-Free Firmware in Future Releases in MakeUseOf

  [1] Choice 5: Change SC for non-free firmware in installer, one installer
  [2] Choice 1: Only one installer, including non-free firmware
  [3] Choice 6: Change SC for non-free firmware in installer, keep both installers
  [4] Choice 2: Recommend installer containing non-free firmware
  [5] Choice 3: Allow presenting non-free installers alongside the free one
  [6] Choice 7: None Of The Above
  [7] Choice 4: Installer with non-free software is not part of Debian

[6] Choice 1: Only one installer, including non-free firmware
[2] Choice 2: Recommend installer containing non-free firmware
[3] Choice 3: Allow presenting non-free installers alongside the free one
[7] Choice 4: Installer with non-free software is not part of Debian
[4] Choice 5: Change SC for non-free firmware in installer, one installer
[1] Choice 6: Change SC for non-free firmware in installer, keep both installers
[5] Choice 7: None Of The Above

[1] Choice 6: Change SC for non-free firmware in installer, keep both installers
[2] Choice 2: Recommend installer containing non-free firmware
[3] Choice 3: Allow presenting non-free installers alongside the free one
[4] Choice 5: Change SC for non-free firmware in installer, one installer
[5] Choice 7: None Of The Above
[6] Choice 1: Only one installer, including non-free firmware
[7] Choice 4: Installer with non-free software is not part of Debian

[update/note] Several people have asked me to back that the Secretary said so. I can refer to four mails: 2022.08.29, 2022.08.30, 2022.09.02, 2022.09.04.

The Debian official media may include firmware that is otherwise not
part of the Debian system to enable use of Debian with hardware that
requires such firmware.

• Cost: it's typically cheaper to fit smaller flash memory (or no flash at all) onto a device. The cost difference may seem small in many cases, but reducing the bill of materials (BOM) even by a few cents can make a substantial difference to the economics of a product. For most vendors, they will have to implement device drivers anyway and it's not difficult to include firmware in that driver.
• Flexibility: it's much easier to change the behaviour of a device by simply changing to a different blob. This can potentially cover lots of different use cases:
  • separating deadlines for hardware and software in manufacturing (drivers and firmware can be written and shipped later);
  • bug fixes and security updates (e.g. CPU microcode changes);
  • changing configuration of a device for different users or products (e.g. potentially different firmware to enable different frequencies on a radio product);
  • changing fundamental device operation (e.g. switching between RAID and JBOD functionality on a disk controller).

• Going back 10 years or so, most computers only needed firmware uploads to make WiFi hardware work.
• A growing number of wired network adapters now demand firmware uploads. Some will work in a limited way but depend on extra firmware to allow advanced features like TCP segmentation offload (TSO). Others will refuse to work at all without a firmware upload.
• More and more graphics adapters now also want firmware uploads to provide any non-basic functions. A simple basic (S)VGA-compatible framebuffer is not enough for most users these days; modern desktops expect 3D acceleration, and a lot of current hardware will not provide that without extra firmware.
• Current generations of common Intel-based laptops also need firmware uploads to make audio work (see the firmware-sof-signed package).

1. Building, testing and publishing two sets of images takes more effort.
2. We don't really want to be providing non-free images at all, from a philosophy point of view. So we mainly promote and advertise the preferred official free images. That can be a cause of confusion for users. We do link to the non-free images in various places, but they're not so easy to find.
3. Using non-free installation media will cause more installations to use non-free software by default. That's not a great story for us, and we may end up with more of our users using non-free software and believing that it's all part of Debian.
4. A number of users and developers complain that we're wasting their time by publishing official images that are just not useful for a lot (a majority?) of users.

1. Keep the existing setup. It's horrible, but maybe it's the best we can do? (I hope not!)
2. We could just stop providing the non-free unofficial images altogether. That's not really a promising route to follow - we'd be making it even harder for users to install our software. While ideologically pure, it's not going to advance the cause of Free Software.
3. We could stop pretending that the non-free images are unofficial, and maybe move them alongside the normal free images so they're published together. This would make them easier to find for people that need them, but is likely to cause users to question why we still make any images without firmware if they're otherwise identical.
4. The images team technically could simply include non-free into the official images, and add firmware packages to the input lists for those images. However, that would still leave us with problem 3 from above (non-free generally enabled on most installations).
5. We could split out the non-free firmware packages into a new non-free-firmware component in the archive, and allow a specific exception only to allow inclusion of those packages on our official media. We would then generate only one set of official media, including those non-free firmware packages. (We've already seen various suggestions in recent years to split up the non-free component of the archive like this, for example into non-free-firmware, non-free-doc, non-free-drivers, etc. Disagreement (bike-shedding?) about the split caused us to not make any progress on this. I believe this project should be picked up and completed. We don't have to make a perfect solution here immediately, just something that works well enough for our needs today. We can always tweak and improve the setup incrementally if that's needed.)

1. Along with adding non-free firmware onto media, when the installer (or live image) runs, we should make it clear exactly which firmware packages have been used/installed to support detected hardware. We could link to docs about each, and maybe also to projects working on Free re-implementations.
2. Add an option at boot to explicitly disable the use of the non-free firmware packages, so that users can choose to avoid them.

• Cyril Brulebois
• Matthew Garrett
• David Leggett
• Martin Michlmayr
• Andy Simpkins
• Neil Williams

• Move your own channels to to Libera or OFTC
• If you have previously been known to be generally around on Freenode, connect to Libera (the continuity network set up by the Freenode staff), and register your usual nick(s) there.
• Disconnect from freenode so that you don't count as one of their users.

Message-ID: <24741.12566.639691.461134@mariner.uk.xensource.com>
From: Ian Jackson <iwj@xenproject.org>
To: xen-devel@lists.xenproject.org
CC: community.manager@xenproject.org
Subject: IRC networks
Date: Wed, 19 May 2021 16:39:02 +0100
Summary:
We have for many years used the Freenode IRC network for real-time
chat about Xen.  Unfortunately, Freenode is undergoing a crisis.
There is a dispute between, on the one hand, Andrew Lee, and on the
other hand, all (or almost all) Freenode volunteer staff.  We must
make a decision.
I have read all the publicly available materials and asked around with
my contacts.  My conclusions:
 * We do not want to continue to use irc.freenode.*.
 * We might want to use libera.chat, but:
 * Our best option is probably to move to OFTC https://www.oftc.net/
Discussion:
Firstly, my starting point.
I have been on IRC since at least 1993.  Currently my main public
networks are OFTC and Freenode.
I do not have any personal involvement with public IRC networks.  Of
the principals in the current Freenode dispute, I have only heard of
one, who is a person I have experience of in a Debian context but have
not worked closely with.
George asked me informally to use my knowledge and contacts to shed
light on the situation.  I decided that, having done my research, I
would report more formally and publicly here rather than just
informally to George.
Historical background:
 * Freenode has had drama before.  In about 2001 OFTC split off from
   Freenode after an argument over governance.  IIRC there was drama
   again in 2006.  Significant proportion of the Free Software world,
   including Debian, now use OFTC.  Debian switched in 2006.
Facts that I'm (now) pretty sure of:
 * Freenode's actual servers run on donated services; that is,
   the hardware is owned by those donating the services, and the
   systems are managed by Freenode volunteers, known as "staff".
 * The freenode domain names are currently registered to a limited
   liability company owned by Andrew Lee (rasengan).
 * At least 10 Freenode staff have quit in protest, writing similar
   resignation letters protesting about Andrew Lee's actions [1].  It
   does not appear that any Andrew Lee has the public support of any
   Freenode staff.
 * Andrew Lee claims that he "owns" Freenode.[2]
 * A large number of channel owners for particular Free Software
   projects who previously used Freenode have said they will switch
   away from Freenode.
Discussion and findings on Freenode:
There is, as might be expected, some murk about who said what to whom
when, what promises were made and/or broken, and so on.  The matter
was also complicated by the leaking earlier this week of draft(s) of
(at least one of) the Freenode staffers' resignation letters.
Andrew Lee has put forward a position statement [2].  A large part of
the thrust of that statement is allegations that the current head of
Freenode staff, tomaw, "forced out" the previous head, christel.  This
allegation is strongly disputed by by all those current (resigning)
Freenode staff I have seen comment.  In any case it does not seem to
be particularly germane; in none of my reading did tomaw seem to be
playing any kind of leading role.  tomaw is not mentioned in the
resignation letters.
Some of the links led to me to logs of discussions on #freenode.  I
read some of these in particular[3].  MB I haven't been able to verify
that these logs have not been tampered with.  Having said that and
taking the logs at face value, I found the rasengan writing there to
be disingenuous and obtuse.
Andrew Lee has been heavily involved in Bitcoin.  Bitcoin is a hive of
scum and villainy, a pyramid scheme, and an environmental disaster,
all rolled into one.  This does not make me think well of Lee.
Additionally, it seems that Andrew Lee has been involved in previous
governance drama involving a different IRC network, Snoonet.
I have come to the very firm conclusion that we should have nothing to
do with Andrew Lee, and avoid using services that he has some
effective control over.
Alternatives:
The departing Freenode staff are setting up a replacement,
"libera.chat".  This is operational but still suffering from teething
problems and of course has a significant load as it deals with an
influx of users on a new setup.
On the staff and trust question: As I say, I haven't heard of any of
the Freenode staff, with one exception.  Unfortunately the one
exception does not inspire confidence in me[4] - although NB that is
only one data point.
On the other hand, Debian has had many many years of drama-free
involvement with OFTC.  OFTC has a formal governance arrangement and
it is associated with Software in the Public Interest.  I notice that
the last few OFTC'[s annual officer elections have been run partly by
Steve McIntyre.  Steve is a friend of mine (and he is a former Debian
Project Leader) and I take his involvement as a good sign.
I recommend that we switch to using OFTC as soon as possible.
Ian.
References:
Starting point for the resigning Freenode staff's side [1]:
  https://gist.github.com/joepie91/df80d8d36cd9d1bde46ba018af497409
Andrew Lee's side [2]:
  https://gist.github.com/realrasengan/88549ec34ee32d01629354e4075d2d48
[3]
https://paste.sr.ht/~ircwright/7e751d2162e4eb27cba25f6f8893c1f38930f7c4
[4] I won't give the name since I don't want to be shitposting.

• Mythic Beasts
• Collabora
• Codethink
• QvarnLabs
• Chris Boot

• There were two talks mainly about Reproducible Builds at DebConf17:
  • Reproducible builds: Status update by Chris Lamb, Holger Levsen, Maria Glukhova, Steven Chamberlain, Vagrant Cascadian, Valerie Young and Ximin Luo.
  • Fun with .buildinfo by Steven Chamberlain.
• Reproducible builds were also mentioned in many other talks including:
  • Using Qubes OS from the POV of a Debian developer by Holger Levsen.
  • Debian for Medical Software by Andr Roth.
  • Let's maintain jenkins.debian.org as a team by Holger Levsen.
  • Signing package contents: why and how by Matthew Garrett.
  • Bits from the DPL by Chris Lamb.
  • Installing Debian by Vagrant Cascadian.
  • in-toto -- Securing supply chains as a whole by Lukas Puehringer.
  • Debian Cloud BoF by Steve McIntyre.
  • Will there be Debian in your next BMW car? by Aigars Mahinovs.
• There has also been substantial activity regarding updating the Debian Policy regarding Reproducible Builds (#844431). This work is still ongoing and will be covered in future issues of this weekly blog.

• Adrian Bunk:
  • #871818 filed against debian-zh-faq.
  • #871907 filed against praat.
• Katsuhiko Nishimra:
  • #871591 filed against llvm-toolchain-snapshot.
• Lucas Nussbaum:
  • #871059 filed against fltk1.3.
  • #871155 filed against brltty.
  • #871159 filed against texlive-extra.
  • #871345 filed against texlive-base.
  • #871349 filed against ispell-uk.
  • #871357 filed against po4a.
• jathan:
  • #870890 filed against apg.

• Bernhard M. Wiedemann:
  • enblend (merged), SOURCE_DATE_EPOCH support
  • systemtap
  • splint (dead project)
  • skelcd-openSUSE

• During our reproducibility testing, Adrian Bunk filed 48 FTBFS bugs this week.

• Mattia Rizzolo:
  • debian/copyright: coalesce some file paragraphs and update information.
  • Bump Standards-Version to 4.0.1, no changes needed.

• Chris Lamb:
  • Test --help output in autopkgtests

• Mattia:
  • Notify the#debian-reproducible-changes IRC channel for unreproducible -> FTBFS transitions.
  • Update squid.conf for all nodes to 5.2.23 (and fixup some).
  • Enable the Munin Squid plugin on the Codethink arm64 nodes as well.
  • Force reconfiguration of Apache and Munin when update_jdn.sh is updated.
• Holger worked on slides for his DebConf17 BoF about migrating to jenkins.debian.org, which affects tests.r-b.o as well.

From: Tec Services <tecservices911@gmail.com>
Date: Wed, 21 Jun 2017 22:30:26 -0700
To: steve@einval.com
Subject: its time for you to retire from debian...unbelievable..your
         the quality guy and fucked up the installer!
i cant ever remember in the hostory of computing someone releasing an installer
that does not work!!
wtf!!!
you need to be retired...due to being retarded..
and that this was dedicated to ian...what a
disaster..you should be ashames..he is probably roling in his grave from shame
right now....