Search Results: "ugs"

21 July 2024

Russell Coker: SE Linux Policy for Dell Management

The recent issue of Windows security software killing computers has reminded me about the issue of management software for Dell systems. I wrote policy for the Dell management programs that extract information from iDRAC and store it in Linux. After the break I ve pasted in the policy. It probably needs some changes for recent software, it was last tested on a PowerEdge T320 and prior to that was used on a PowerEdge R710 both of which are old hardware and use different management software to the recent hardware. One would hope that the recent software would be much better but usually such hope is in vain. I deliberately haven t submitted this for inclusion in the reference policy because it s for proprietary software and also it permits many operations that we would prefer not to permit. The policy is after the break because it s larger than you want on a Planet feed. But first I ll give a few selected lines that are bad in a noteworthy way:
  1. sys_admin means the ability to break everything
  2. dac_override means break Unix permissions
  3. mknod means a daemon creates devices due to a lack of udev configuration
  4. sys_rawio means someone didn t feel like writing a device driver, maintaining a device driver for DKMS is hard and getting a driver accepted upstream requires writing quality code, in any case this is a bad sign.
  5. self:lockdown is being phased out, but used to mean bypassing some integrity protections, that would usually be related to sys_rawio or similar.
  6. dev_rx_raw_memory is bad, reading raw memory allows access to pretty much everything and execute of raw memory is something I can t imagine a good use for, the Reference Policy doesn t use this anywhere!
  7. dev_rw_generic_chr_files usually means a lack of udev configuration as udev should do that.
  8. storage_raw_write_fixed_disk shouldn t be needed for this sort of thing, it doesn t do anything that involves managing partitions.
Now without network access or other obvious ways of remote control this level of access while excessive isn t necessarily going to allow bad things to happen due to outside attack. But if there are bugs in the software there s nothing to stop it from giving the worst results.
allow dell_datamgrd_t self:capability   dac_override dac_read_search mknod sys_rawio sys_admin  ;
allow dell_datamgrd_t self:lockdown integrity;
dev_rx_raw_memory(dell_datamgrd_t)
dev_rw_generic_chr_files(dell_datamgrd_t)
dev_rw_ipmi_dev(dell_datamgrd_t)
dev_rw_sysfs(dell_datamgrd_t)
storage_raw_read_fixed_disk(dell_datamgrd_t)
storage_raw_write_fixed_disk(dell_datamgrd_t)
allow dellsrvadmin_t self:lockdown integrity;
allow dellsrvadmin_t self:capability   sys_admin sys_rawio  ;
dev_read_raw_memory(dellsrvadmin_t)
dev_rw_sysfs(dellsrvadmin_t)
dev_rx_raw_memory(dellsrvadmin_t)
The best thing that Dell could do for their customers is to make this free software and allow the community to fix some of these issues.
Here is dellsrvadmin.te:
policy_module(dellsrvadmin,1.0.0)
require  
  type dmidecode_exec_t;
  type udev_t;
  type device_t;
  type event_device_t;
  type mon_local_test_t;
 
type dellsrvadmin_t;
type dellsrvadmin_exec_t;
init_daemon_domain(dellsrvadmin_t, dellsrvadmin_exec_t)
type dell_datamgrd_t;
type dell_datamgrd_exec_t;
init_daemon_domain(dell_datamgrd_t, dell_datamgrd_t)
type dellsrvadmin_var_t;
files_type(dellsrvadmin_var_t)
domain_transition_pattern(udev_t, dellsrvadmin_exec_t, dellsrvadmin_t)
modutils_domtrans(dellsrvadmin_t)
allow dell_datamgrd_t device_t:dir rw_dir_perms;
allow dell_datamgrd_t device_t:chr_file create;
allow dell_datamgrd_t event_device_t:chr_file   read write  ;
allow dell_datamgrd_t self:process signal;
allow dell_datamgrd_t self:fifo_file rw_file_perms;
allow dell_datamgrd_t self:sem create_sem_perms;
allow dell_datamgrd_t self:capability   dac_override dac_read_search mknod sys_rawio sys_admin  ;
allow dell_datamgrd_t self:lockdown integrity;
allow dell_datamgrd_t self:unix_dgram_socket create_socket_perms;
allow dell_datamgrd_t self:netlink_route_socket r_netlink_socket_perms;
modutils_domtrans(dell_datamgrd_t)
can_exec(dell_datamgrd_t, dmidecode_exec_t)
allow dell_datamgrd_t dellsrvadmin_var_t:dir rw_dir_perms;
allow dell_datamgrd_t dellsrvadmin_var_t:file manage_file_perms;
allow dell_datamgrd_t dellsrvadmin_var_t:lnk_file read;
allow dell_datamgrd_t dellsrvadmin_var_t:sock_file manage_file_perms;
kernel_read_network_state(dell_datamgrd_t)
kernel_read_system_state(dell_datamgrd_t)
kernel_search_fs_sysctls(dell_datamgrd_t)
kernel_read_vm_overcommit_sysctl(dell_datamgrd_t)
# for /proc/bus/pci/*
kernel_write_proc_files(dell_datamgrd_t)
corecmd_exec_bin(dell_datamgrd_t)
corecmd_exec_shell(dell_datamgrd_t)
corecmd_shell_entry_type(dell_datamgrd_t)
dev_rx_raw_memory(dell_datamgrd_t)
dev_rw_generic_chr_files(dell_datamgrd_t)
dev_rw_ipmi_dev(dell_datamgrd_t)
dev_rw_sysfs(dell_datamgrd_t)
files_search_tmp(dell_datamgrd_t)
files_read_etc_files(dell_datamgrd_t)
files_read_etc_symlinks(dell_datamgrd_t)
files_read_usr_files(dell_datamgrd_t)
logging_search_logs(dell_datamgrd_t)
miscfiles_read_localization(dell_datamgrd_t)
storage_raw_read_fixed_disk(dell_datamgrd_t)
storage_raw_write_fixed_disk(dell_datamgrd_t)
can_exec(mon_local_test_t, dellsrvadmin_exec_t)
allow mon_local_test_t dellsrvadmin_var_t:dir search;
allow mon_local_test_t dellsrvadmin_var_t:file read_file_perms;
allow mon_local_test_t dellsrvadmin_var_t:file setattr;
allow mon_local_test_t dellsrvadmin_var_t:sock_file write;
allow mon_local_test_t dell_datamgrd_t:unix_stream_socket connectto;
allow mon_local_test_t self:sem   create read write destroy unix_write  ;
allow dellsrvadmin_t self:process signal;
allow dellsrvadmin_t self:lockdown integrity;
allow dellsrvadmin_t self:sem create_sem_perms;
allow dellsrvadmin_t self:fifo_file rw_file_perms;
allow dellsrvadmin_t self:packet_socket create;
allow dellsrvadmin_t self:unix_stream_socket   connectto create_stream_socket_perms  ;
allow dellsrvadmin_t self:capability   sys_admin sys_rawio  ;
dev_read_raw_memory(dellsrvadmin_t)
dev_rw_sysfs(dellsrvadmin_t)
dev_rx_raw_memory(dellsrvadmin_t)
allow dellsrvadmin_t dellsrvadmin_var_t:dir rw_dir_perms;
allow dellsrvadmin_t dellsrvadmin_var_t:file manage_file_perms;
allow dellsrvadmin_t dellsrvadmin_var_t:lnk_file read;
allow dellsrvadmin_t dellsrvadmin_var_t:sock_file write;
allow dellsrvadmin_t dell_datamgrd_t:unix_stream_socket connectto;
kernel_read_network_state(dellsrvadmin_t)
kernel_read_system_state(dellsrvadmin_t)
kernel_search_fs_sysctls(dellsrvadmin_t)
kernel_read_vm_overcommit_sysctl(dellsrvadmin_t)
corecmd_exec_bin(dellsrvadmin_t)
corecmd_exec_shell(dellsrvadmin_t)
corecmd_shell_entry_type(dellsrvadmin_t)
files_read_etc_files(dellsrvadmin_t)
files_read_etc_symlinks(dellsrvadmin_t)
files_read_usr_files(dellsrvadmin_t)
logging_search_logs(dellsrvadmin_t)
miscfiles_read_localization(dellsrvadmin_t)
Here is dellsrvadmin.fc:
/opt/dell/srvadmin/sbin/.*        --        gen_context(system_u:object_r:dellsrvadmin_exec_t,s0)
/opt/dell/srvadmin/sbin/dsm_sa_datamgrd        --        gen_context(system_u:object_r:dell_datamgrd_t,s0)
/opt/dell/srvadmin/bin/.*        --        gen_context(system_u:object_r:dellsrvadmin_exec_t,s0)
/opt/dell/srvadmin/var(/.*)?                        gen_context(system_u:object_r:dellsrvadmin_var_t,s0)
/opt/dell/srvadmin/etc/srvadmin-isvc/ini(/.*)?        gen_context(system_u:object_r:dellsrvadmin_var_t,s0)

17 July 2024

Dirk Eddelbuettel: Rcpp 1.0.13 on CRAN: Some Updates

rcpp logo The Rcpp Core Team is once again pleased to announce a new release (now at 1.0.13) of the Rcpp package. It arrived on CRAN earlier today, and has since been uploaded to Debian. Windows and macOS builds should appear at CRAN in the next few days, as will builds in different Linux distribution and of course r2u should catch up tomorrow too. The release was uploaded last week, but not only does Rcpp always gets flagged because of the grandfathered .Call(symbol) but CRAN also found two packages regressing which then required them to take five days to get back to us. One issue was known; another did not reproduce under our tests against over 2800 reverse dependencies leading to the eventual release today. Yay. Checks are good and appreciated, and it does take time by humans to review them. This release continues with the six-months January-July cycle started with release 1.0.5 in July 2020. As a reminder, we do of course make interim snapshot dev or rc releases available via the Rcpp drat repo as well as the r-universe page and repo and strongly encourage their use and testing I run my systems with these versions which tend to work just as well, and are also fully tested against all reverse-dependencies. Rcpp has long established itself as the most popular way of enhancing R with C or C++ code. Right now, 2867 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 256 in BioConductor. On CRAN, 13.6% of all packages depend (directly) on Rcpp, and 59.9% of all compiled packages do. From the cloud mirror of CRAN (which is but a subset of all CRAN downloads), Rcpp has been downloaded 86.3 million times. The two published papers (also included in the package as preprint vignettes) have, respectively, 1848 (JSS, 2011) and 324 (TAS, 2018) citations, while the the book (Springer useR!, 2013) has another 641. This release is incremental as usual, generally preserving existing capabilities faithfully while smoothing our corners and / or extending slightly, sometimes in response to changing and tightened demands from CRAN or R standards. The move towards a more standardized approach for the C API of R leads to a few changes; Kevin did most of the PRs for this. Andrew Johnsom also provided a very nice PR to update internals taking advantage of variadic templates. The full list below details all changes, their respective PRs and, if applicable, issue tickets. Big thanks from all of us to all contributors!

Changes in Rcpp release version 1.0.13 (2024-07-11)
  • Changes in Rcpp API:
    • Set R_NO_REMAP if not already defined (Dirk in #1296)
    • Add variadic templates to be used instead of generated code (Andrew Johnson in #1303)
    • Count variables were switches to size_t to avoid warnings about conversion-narrowing (Dirk in #1307)
    • Rcpp now avoids the usage of the (non-API) DATAPTR function when accessing the contents of Rcpp Vector objects where possible. (Kevin in #1310)
    • Rcpp now emits an R warning on out-of-bounds Vector accesses. This may become an error in a future Rcpp release. (Kevin in #1310)
    • Switch VECTOR_PTR and STRING_PTR to new API-compliant RO variants (Kevin in #1317 fixing #1316)
  • Changes in Rcpp Deployment:
    • Small updates to the CI test containers have been made (#1304)

Thanks to my CRANberries, you can also look at a diff to the previous release Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues). If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

14 July 2024

Russ Allbery: DocKnot 8.0.1

DocKnot is my static web site generator, with some additional features for managing software releases. This release fixes some bugs in the newly-added conversion of text to HTML that were due to my still-incomplete refactoring of that code. It still uses some global variables, and they were leaking between different documents and breaking the formatting. It also fixes consistency problems with how the style parameter in *.spin files was interpreted, and fixes some incorrect docknot update-spin behavior. You can get the latest version from CPAN or from the DocKnot distribution page.

12 July 2024

Reproducible Builds: Reproducible Builds in June 2024

Welcome to the June 2024 report from the Reproducible Builds project! In our reports, we outline what we ve been up to over the past month and highlight news items in software supply-chain security more broadly. As always, if you are interested in contributing to the project, please visit our Contribute page on our website. Table of contents:
  1. Next Reproducible Builds Summit dates announced
  2. GNU Guix patch review session for reproducibility
  3. New reproducibility-related academic papers
  4. Misc development news
  5. Website updates
  6. Reproducibility testing framework


Next Reproducible Builds Summit dates announced We are very pleased to announce the upcoming Reproducible Builds Summit, set to take place from September 17th 19th 2024 in Hamburg, Germany. We are thrilled to host the seventh edition of this exciting event, following the success of previous summits in various iconic locations around the world, including Venice, Marrakesh, Paris, Berlin and Athens. Our summits are a unique gathering that brings together attendees from diverse projects, united by a shared vision of advancing the Reproducible Builds effort. During this enriching event, participants will have the opportunity to engage in discussions, establish connections and exchange ideas to drive progress in this vital field. Our aim is to create an inclusive space that fosters collaboration, innovation and problem-solving. If you re interesting in joining us this year, please make sure to read the event page which has more details about the event and location. We are very much looking forward to seeing many readers of these reports there.

GNU Guix patch review session for reproducibility Vagrant Cascadian will holding a Reproducible Builds session as part of the monthly Guix patch review series on July 11th at 17:00 UTC. These online events are intended to encourage everyone everyone becoming a patch reviewer and the goal of reviewing patches is to help Guix project accept contributions while maintaining our quality standards and learning how to do patch reviews together in a friendly hacking session.

Development news In Debian this month, 4 reviews of Debian packages were added, 11 were updated and 14 were removed this month adding to our knowledge about identified issues. Only one issue types was updated, though, explaining that we don t vary the build path anymore.
On our mailing list this month, Bernhard M. Wiedemann wrote that whilst he had previously collected issues that introduce non-determinism he has now moved on to discuss about mitigations , in the sense of how can we avoid whole categories of problem without patching an infinite number of individual packages . In addition, Janneke Nieuwenhuizen announced the release of two versions of GNU Mes. [ ][ ]
In openSUSE news, Bernhard M. Wiedemann published another report for that distribution.
In NixOS, with the 24.05 release out, it was again validated that our minimal ISO is reproducible by building it on a virtual machine with no access to the binary cache.
What s more, we continued to write patches in order to fix specific reproducibility issues, including Bernhard M. Wiedemann writing three patches (for qutebrowser, samba and systemd), Chris Lamb filing Debian bug #1074214 against the fastfetch package and Arnout Engelen proposing fixes to refind and for the Scala compiler [ .
Lastly, diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb uploaded two versions (270 and 271) to Debian, and made the following changes as well:
  • Drop Build-Depends on liblz4-tool in order to fix Debian bug #1072575. [ ]
  • Update tests to support zipdetails version 4.004 that is shipped with Perl 5.40. [ ]

Website updates There were a number of improvements made to our website this month, including Akihiro Suda very helpfully making the <h4> elements more distinguishable from the <h3> level [ ][ ] as well as adding a guide for Dockerfile reproducibility [ ]. In addition Fay Stegerman added two tools, apksigcopier and reproducible-apk-tools, to our Tools page.

Reproducibility testing framework The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In June, a number of changes were made by Holger Levsen, including:
  • Marking the virt(32 64)c-armhf nodes as down. [ ]
  • Granting a developer access to the osuosl4 node in order to debug a regression on the ppc64el architecture. [ ]
  • Granting a developer access to the osuosl4 node. [ ][ ]
In addition, Mattia Rizzolo re-aligned the /etc/default/jenkins file with changes performed upstream [ ] and changed how configuration files are handled on the rb-mail1 host. [ ], whilst Vagrant Cascadian documented the failure of the virt32c and virt64c nodes after initial investigation [ ].

If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

11 July 2024

Russ Allbery: podlators v6.0.0

podlators is the collection of Perl modules and front-end scripts that convert POD documentation to *roff manual pages or text, possibly with formatting intended for pagers. This release continues the simplifications that I've been doing in the last few releases and now uniformly escapes - characters and single quotes, disabling all special interpretation by *roff formatters and dropping the heuristics that were used in previous versions to try to choose between possible interpretations of those characters. I've come around to the position that POD simply is not semantically rich enough to provide sufficient information to successfully make a more nuanced conversion, and these days Unicode characters are available for authors who want to be more precise. This version also drops support for Perl versions prior to 5.12 and switches to semantic versioning for all modules. I've added a v prefix to the version number, since that is the convention for Perl module versions that use semantic versioning. This release also works around some changes to the man macros in groff 1.23.0 to force persistent ragged-right justification when formatted with nroff and fixes a variety of other bugs. You can get the latest release from CPAN or from the podlators distribution page.

10 July 2024

Russell Coker: Computer Adavances in the Last Decade

I wrote a comment on a social media post where someone claimed that there s no computer advances in the last 12 years which got long so it s worth a blog post. In the last decade or so new laptops have become cheaper than new desktop PCs. USB-C has taken over for phones and for laptop charging so all recent laptops support USB-C docks and monitors with USB-C docks built in have become common. 4K monitors have become cheap and common and higher than 4K is cheap for some use cases such as ultra wide. 4K TVs are cheap and TVs with built-in Android computers for playing internet content are now standard. For most use cases spinning media hard drives are obsolete, SSDs large enough for all the content most people need to store are cheap. We have gone from gigabit Ethernet being expensive to 2.5 gigabit being cheap. 12 years ago smart phones were very limited and every couple of years there would be significant improvements. Since about 2018 phones have been capable of doing most things most people want. 5yo Android phones can run the latest apps and take high quality pics. Any phone that supports VoLTE will be good for another 5+ years if it has security support. Phones without security support still work and are quite usable apart from being insecure. Google and Samsung have significantly increased their minimum security support for their phones and the GKI project from Google makes it easier for smaller vendors to give longer security support. There are a variety of open Android projects like LineageOS which give longer security support on a variety of phones. If you deliberately choose a phone that is likely to be well supported by projects like LineageOS (which pretty much means just Pixel phones) then you can expect to be able to actually use it when it is 10 years old. Compare this to the Samsung Galaxy S3 released in 2012 which was a massive improvement over the original Galaxy S (the S2 felt closer to the S than the S3). The Samsung Galaxy S4 released in 2013 was one of the first phones to have FullHD resolution which is high enough that most people can t easily recognise the benefits of higher resolution. It wasn t until 2015 that phones with 4G of RAM became common which is enough that for most phone use it s adequate today. Now that 16G of RAM is affordable in laptops running more secure OSs like Qubes is viable for more people. Even without Qubes, OS security has been improving a lot with better compiler features, new languages like Rust, and changes to software design and testing. Containers are being used more but we still aren t getting all the benefits of that. TPM has become usable in the last few years and we are only starting to take advantage of what it can offer. In 2012 BTRFS was still at an early stage of development and not many people wanted to use it in production, I was using it in production then and while I didn t lose any data from bugs I did have some downtime because of BTRFS issues. Now BTRFS is quite solid for server use. DDR4 was released in 2014 and gave significant improvements over DDR3 for performance and capacity. My home workstation now has 256G of DDR4 which wasn t particularly expensive while the previous biggest system I owned had 96G of DDR3 RAM. Now DDR5 is available to again increase performance and size while also making DDR4 cheap on the second hand market. This isn t a comprehensive list of all advances in the computer industry over the last 12 years or so, it s just some things that seem particularly noteworthy to me. Please comment about what you think are the most noteworthy advances I didn t mention.

Freexian Collaborators: Debian Contributions: YubiHSM packaging, unschroot, live-patching, and more! (by Stefano Rivera)

Debian Contributions: 2024-06 Contributing to Debian is part of Freexian s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

YubiHSM packaging, by Colin Watson Freexian is starting to use YubiHSM devices (hardware security modules) as part of some projects, and we wanted to have the supporting software directly in Debian rather than needing to use third-party repositories. Since Yubico publish everything we need under free software licences, Colin packaged yubihsm-connector, yubihsm-shell, and python-yubihsm from https://developers.yubico.com/, in some cases based partly on the upstream packaging, and got them all into Debian unstable. Backports to bookworm will be forthcoming once they ve all reached testing.

unschroot by Helmut Grohne Following an in-person discussion at MiniDebConf Berlin, Helmut attempted splitting the containment functionality of sbuild --chroot-mode=unshare into a dedicated tool interfacing with sbuild as a variant of --chroot-mode=schroot providing a sufficiently compatible interface. While this seemed technically promising initially, a discussion on debian-devel indicated a desire to rely on an existing container runtime such as podman instead of using another Debian-specific tool with unclear long term maintenance. None of the existing container runtimes meet the specific needs of sbuild, so further advancing this matter implies a compromise one way or another.

Linux live-patching, by Santiago Ruano Rinc n In collaboration with Emmanuel Arias, Santiago is working on the development of linux live-patching for Debian. For the moment, this is in an exploratory phase, that includes how to handle the different patches that will need to be provided. kpatch could help significantly in this regard. However, kpatch was removed from unstable because there are some RC bugs affecting the version that was present in Debian unstable. Santiago packaged the most recent upstream version (0.9.9) and filed an Intent to Salvage bug. Santiago is waiting for an ACK by the maintainer, and will upload to unstable after July 10th, following the package salvaging rules. While kpatch 0.9.9 fixes the main issues, it still needs some work to properly support Debian and the Linux kernel versions packaged in our distribution. More on this in the report next month.

Salsa CI, by Santiago Ruano Rinc n The work by Santiago in Salsa CI this month includes a merge request to ease testing how the production images are built from the changes introduced by future merge requests. By default, the pipelines triggered by a merge request build a subset of the images built for production, to reduce the use of resources, and because most of the time the subset of staging images is enough to test the proposed modifications. However, sometimes it is needed to test how the full set of production images is built, and the above mentioned MR helps to do that. The changes include documentation, so hopefully this will make it easier to test future contributions. Also, for being able to include support for RISC-V, Salsa CI needs to replace kaniko as the tool used to build the images. Santiago tested buildah, but there are some issues when pushing built images for non-default platform architectures (i386, armhf, armel) to the container registry. Santiago will continue to work on this to find a solution.

Miscellaneous contributions
  • Stefano Rivera prepared updates for a number of Python modules.
  • Stefano uploaded the latest point release of Python 3.12 and the latest Python 3.13 beta. Both uncovered upstream regressions that had to be addressed.
  • Stefano worked on preparations for DebConf 24.
  • Stefano helped SPI to reconcile their financial records for DebConf 23.
  • Colin did his usual routine work on the Python team, upgrading 36 packages to new upstream versions (including fixes for four CVEs in python-aiohttp), fixing RC bugs in ipykernel, ipywidgets, khard, and python-repoze.sphinx.autointerface, and packaging zope.deferredimport which was needed for a new upstream version of python-persistent.
  • Colin removed the user_readenv option from OpenSSH s PAM configuration (#1018260), and prepared a release note.
  • Thorsten Alteholz uploaded a new upstream version of cups.
  • Nicholas Skaggs updated xmacro to support reproducible builds (#1014428), DEP-3 and DEP-5 compatibility, along with utilizing hardening build flags. Helmut supported and uploaded package.
  • As a result of login having become non-essential, Helmut uploaded debvm to unstable and stable and fixed a crossqa.debian.net worker.
  • Santiago worked on the Content Team activities for DebConf24. Together with other DebConf25 team members, Santiago wrote a document for the head of the venue to describe the project of the conference.

9 July 2024

Simon Josefsson: Towards Idempotent Rebuilds?

After rebuilding all added/modified packages in Trisquel, I have been circling around the elephant in the room: 99% of the binary packages in Trisquel comes from Ubuntu, which to a large extent are built from Debian source packages. Is it possible to rebuild the official binary packages identically? Does anyone make an effort to do so? Does anyone care about going through the differences between the official package and a rebuilt version? Reproducible-build.org s effort to track reproducibility bugs in Debian (and other systems) is amazing. However as far as I know, they do not confirm or deny that their rebuilds match the official packages. In fact, typically their rebuilds do not match the official packages, even when they say the package is reproducible, which had me surprised at first. To understand why that happens, compare the buildinfo file for the official coreutils 9.1-1 from Debian bookworm with the buildinfo file for reproducible-build.org s build and you will see that the SHA256 checksum does not match, but still they declare it as a reproducible package. As far as I can tell of the situation, the purpose of their rebuilds are not to say anything about the official binary build, instead the purpose is to offer a QA service to maintainers by performing two builds of a package and declaring success if both builds match. I have felt that something is lacking, and months have passed and I haven t found any project that address the problem I am interested in. During my earlier work I created a project called debdistreproduce which performs rebuilds of the difference between two distributions in a GitLab pipeline, and display diffoscope output for further analysis. A couple of days ago I had the idea of rewriting it to perform rebuilds of a single distribution. A new project debdistrebuild was born and today I m happy to bless it as version 1.0 and to announces the project! Debdistrebuild has rebuilt the top-50 popcon packages from Debian bullseye, bookworm and trixie, on amd64 and arm64, as well as Ubuntu jammy and noble on amd64, see the summary status page for links. This is intended as a proof of concept, to allow people experiment with the concept of doing GitLab-based package rebuilds and analysis. Compare how Guix has the guix challenge command. Or I should say debdistrebuild has attempted to rebuild those distributions. The number of identically built packages are fairly low, so I didn t want to waste resources building the rest of the archive until I understand if the differences are due to consequences of my build environment (plain apt-get build-dep followed by dpkg-buildpackage in a fresh container), or due to some real difference. Summarizing the results, debdistrebuild is able to rebuild 34% of Debian bullseye on amd64, 36% of bookworm on amd64, 32% of bookworm on arm64. The results for trixie and Ubuntu are disappointing, below 10%. So what causes my rebuilds to be different from the official rebuilds? Some are trivial like the classical problem of varying build paths, resulting in a different NT_GNU_BUILD_ID causing a mismatch. Some are a bit strange, like a subtle difference in one of perl s headers file. Some are due to embedded version numbers from a build dependency. Several of the build logs and diffoscope outputs doesn t make sense, likely due to bugs in my build scripts, especially for Ubuntu which appears to strip translations and do other build variations that I don t do. In general, the classes of reproducibility problems are the expected. Some are assembler differences for GnuPG s gpgv-static, likely triggered by upload of a new version of gcc after the original package was built. There are at least two ways to resolve that problem: either use the same version of build dependencies that were used to produce the original build, or demand that all packages that are affected by a change in another package are rebuilt centrally until there are no more differences. The current design of debdistrebuild uses the latest version of a build dependency that is available in the distribution. We call this a idempotent rebuild . This is usually not how the binary packages were built originally, they are often built against earlier versions of their build dependency. That is the situation for most binary distributions. Instead of using the latest build dependency version, higher reproducability may be achieved by rebuilding using the same version of the build dependencies that were used during the original build. This requires parsing buildinfo files to find the right version of the build dependency to install. We believe doing so will lead to a higher number of reproducibly built packages. However it begs the question: can we rebuild that earlier version of the build dependency? This circles back to really old versions and bootstrappable builds eventually. While rebuilding old versions would be interesting on its own, we believe that is less helpful for trusting the latest version and improving a binary distribution: it is challenging to publish a new version of some old package that would fix a reproducibility bug in another package when used as a build dependency, and then rebuild the later packages with the modified earlier version. Those earlier packages were already published, and are part of history. It may be that ultimately it will no longer be possible to rebuild some package, because proper source code is missing (for packages using build dependencies that were never part of a release); hardware to build a package could be missing; or that the source code is no longer publicly distributable. I argue that getting to 100% idempotent rebuilds is an interesting goal on its own, and to reach it we need to start measure idempotent rebuild status. One could conceivable imagine a way to rebuild modified versions of earlier packages, and then rebuild later packages using the modified earlier packages as build dependencies, for the purpose of achieving higher level of reproducible rebuilds of the last version, and to reach for bootstrappability. However, it may be still be that this is insufficient to achieve idempotent rebuilds of the last versions. Idempotent rebuilds are different from a reproducible build (where we try to reproduce the build using the same inputs), and also to bootstrappable builds (in which all binaries are ultimately built from source code). Consider a cycle where package X influence the content of package Y, which in turn influence the content of package X. These cycles may involve several packages, and it is conceivable that a cycle could be circular and infinite. It may be difficult to identify these chains, and even more difficult to break them up, but this effort help identify where to start looking for them. Rebuilding packages using the same build dependency versions as were used during the original build, or rebuilding packages using a bootsrappable build process, both seem orthogonal to the idempotent rebuild problem. Our notion of rebuildability appears thus to be complementary to reproducible-builds.org s definition and bootstrappable.org s definition. Each to their own devices, and Happy Hacking! Addendum about terminology: With idempotent rebuild I am talking about a rebuild of the entire operating system, applied to itself. Compare how you build the latest version of the GNU C Compiler: it first builds itself using whatever system compiler is available (often an earlier version of gcc) which we call step 1. Then step 2 is to build a copy of itself using the compiler built in step 1. The final step 3 is to build another copy of itself using the compiler from step 2. Debian, Ubuntu etc are at step 1 in this process right now. The output of step 2 and step 3 ought to be bit-by-bit identical, or something is wrong. The comparison between step 2 and 3 is what I refer to with an idempotent rebuild. Of course, most packages aren t a compiler that can compile itself. However entire operating systems such as Trisquel, PureOS, Ubuntu or Debian are (hopefully) a self-contained system that ought to be able to rebuild itself to an identical copy. Or something is amiss. The reproducible build and bootstrappable build projects are about improve the quality of step 1. The property I am interested is the identical rebuild and comparison in step 2 and 3. I feel the word idempotent describes the property I m interested in well, but I realize there may be better ways to describe this. Ideas welcome!

8 July 2024

Thorsten Alteholz: My Debian Activities in June 2024

FTP master This month I accepted 270 and rejected 23 packages. The overall number of packages that got accepted was 279.

Debian LTS This was my hundred-twentieth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. During my allocated time I uploaded or worked on: This month handling of the CVE of cups was a bit messy. After lifting the embargo of the CVE, a published patch did not work with all possible combinations of the configuration. In other words, in cases of having only one local domain socket configured, the cupsd did not start and failed with a strange error. Anyway, upstream published a new set of patches, which made cups work again. Unfortunately this happended just before the latest point release for Bullseye and Bookworm, so that the new packages did not make it into the release, but stopped in the corresponding p-u-queues: stable-p-u and old-p-u. I also continued to work on tiff and last but not least did a week of FD and attended the monthly LTS/ELTS meeting. Debian ELTS This month was the seventy-first ELTS month. During my allocated time I tried to upload a new version of cups for Jessie and Stretch. Unfortunately this was stopped due to an autopkgtest error, which I could not reproduce yet. I also wanted to finally upload a fixed version of exim4. Unfortunately this was stopped due to lots of CI-jobs for Buster. Updates for Buster are now also availble from ELTS, so some stuff had to prepared before the actual switch end of June. Additionally everything was delayed due to a crash of the CI worker. All in all this month was rather ill-fated. At least the exim4 upload will happen/already happened in July. I also continued to work on an update for libvirt, did a week of FD and attended the LTS/ELTS meeting. Debian Printing This month I uploaded new upstream or bugfix versions of: This work is generously funded by Freexian! Debian Astro This month I uploaded a new upstream or bugfix version of: All of those uploads are somehow related to /usr-move. Debian IoT This month I uploaded new upstream or bugfix versions of: Debian Mobcom The following packages have been prepared by the GSoC student Nathan: misc This month I uploaded new upstream or bugfix versions of: Here as well all uploads are somehow related to /usr-move

7 July 2024

Russ Allbery: Review: Welcome to Boy.Net

Review: Welcome to Boy.Net, by Lyda Morehouse
Series: Earth's Shadow #1
Publisher: Wizard's Tower Press
Copyright: April 2024
ISBN: 1-913892-71-9
Format: Kindle
Pages: 355
Welcome to Boy.Net is a science fiction novel with cyberpunk vibes, the first of a possible series. Earth is a largely abandoned wasteland. Humanity has survived in the rest of the solar system and spread from Earth's moon to the outer planets. Mars is the power in the inner system, obsessed with all things Earth and effectively run by the Earth Nations' Peacekeeping Force, the ENForcers. An ENForcer soldier is raised in a creche from an early age, implanted with cybernetic wetware and nanite enhancements, and extensively trained to be an elite fighting unit. As befits a proper military, every ENForcer is, of course, male. The ENForcers thought Lucia Del Toro was a good, obedient soldier. They also thought she was a man. They were wrong about those and many other things. After her role in an atrocity that named her the Scourge of New Shanghai, she went AWOL and stole her command ship. Now she and her partner/girlfriend Hawk, a computer hacker from Luna, make a living with bounty hunting jobs in the outer system. The ENForcers rarely cross the asteroid belt; the United Miners see to that. The appearance of an F-class ENForcer battle cruiser in Jupiter orbit is a very unpleasant surprise. Lucia and Hawk hope it has nothing to do with them. That hope is dashed when ENForcers turn up in the middle of their next job: a bounty to retrieve an AI eye. I first found Lyda Morehouse via her AngeLINK cyberpunk series, the last of which was published in 2011. Since then, she's been writing paranormal romance and urban fantasy as Tate Hallaway. This return to science fiction is an adventure with trickster hackers, throwback anime-based cowboy bars, tense confrontations with fascist thugs, and unexpected mutual aid, but its core is a cyberpunk look at the people who are unwilling or unable to follow the rules of social conformity. Gender conformity, specifically. Once you understand what this book is about, Welcome to Boy.Net is a great title, but I'm not sure it serves its purpose as a marketing tool. This is not the book that I would have expected from that title in isolation, and I'm a bit worried that people who would like it might pass it by. Inside the story, Boy.Net is the slang term for the cybernetic network that links all ENForcers. If this were the derogatory term used by people outside the ENForcers, I could see it, but it's what the ENForcers themselves call it. That left me with a few suspension of disbelief problems, since the sort of macho assholes who are this obsessed with male gender conformance usually consider "boys" to be derogatory and wouldn't call their military cybernetic network something that sounds that belittling, even as a joke. It would be named after some sort of Orwellian reference to freedom, or something related to violence, dominance, brutality, or some other "traditional male" virtue. But although this term didn't work for me as world-building, it's a beautiful touch thematically. What Morehouse is doing here is the sort of concretized metaphor that science fiction is so good at: an element of world-building that is both an analogy for something the reader is familiar with and is also a concrete piece of world background that follows believable rules and can be manipulated by the characters. Boy.Net is trying to reconnect to Lucia against her will. If it succeeds, it will treat the body modifications she's made as damage and try to reverse all of them, attempting to convert her back to the model of an ENForcer. But it is also a sharp metaphor for how gender roles are enforced in our world: a child assigned male is connected to a pervasive network of gender expectations and is programmed, shaped, and monitored to match the social role of a boy. Even if they reject those expectations, the gender role keeps trying to reconnect and convert them back. I really enjoyed Morehouse's handling of the gender dynamics. It's an important part of the plot, but it's not the only thing going on or the only thing the characters think about. Lucia is occasionally caught by surprise by well-described gender euphoria, but mostly gender is something other people keep trying to impose on her because they're obsessed with forcing social conformity. The rest of the book is a fun romp with a few memorable characters and a couple of great moments with unexpected allies. Hawk and Lucia have an imperfect but low drama relationship that features a great combination of insight and the occasional misunderstanding. It's the kind of believable human relationship that I don't see very much in science fiction, written with the comfortable assurance of an author with over a dozen books under her belt. Some of the supporting characters are also excellent, including a non-binary deaf hacker that I wish had been a bit more central to the story. This is not the greatest science fiction novel I've read, but it was entertaining throughout and kept me turning the pages. Recommended if you want some solar-system cyberpunk in your life. Welcome to Boy.Net reaches a conclusion of sorts, but there's an obvious hook for a sequel and a lot of room left for more stories. I hope enough people buy this book so that I can read it. Rating: 7 out of 10

2 July 2024

Bits from Debian: Bits from the DPL

Dear Debian community, Statement on Daniel Pocock The Debian project has successfully taken action to secure its trademarks and interests worldwide, as detailed in our press statement. I would like to personally thank everyone in the community who was involved in this process. I would have loved for you all to have spent your volunteer time on more fruitful things. Debian Boot team might need help I think I've identified the issue that finally motivated me to contact our teams: for a long time, I have had the impression that Debian is driven by several "one-person teams" (to varying extents of individual influence and susceptibility to burnout). As DPL, I see it as my task to find ways to address this issue and provide support. I received private responses from Debian Boot team members, which motivated me to kindly invite volunteers to some prominent and highly visible fields of work that you might find personally challenging. I recommend subscribing to the Debian Boot mailing list to see where you might be able to provide assistance. /usrmerge Helmut Grohne confirmed that the last remaining packages shipping aliased files inside the package set relevant to debootstrap were uploaded. Thanks a lot for Helmut and all contributors that helped to implement DEP17. Contacting more teams I'd like to repeat that I've registered a BoF for DebConf24 in Busan with the following description: This BoF is an attempt to gather as much as possible teams inside Debian to exchange experiences, discuss workflows inside teams, share their ways to attract newcomers etc. Each participant team should prepare a short description of their work and what team roles ( openings ) they have for new contributors. Even for delegated teams (membership is less fluid), it would be good to present the team, explain what it takes to be a team member, and what steps people usually go to end up being invited to participate. Some other teams can easily absorb contributions from salsa MRs, and at some point people get commit access. Anyway, the point is that we work on the idea that the pathway to become a team member becomes more clear from an outsider point-of-view. I'm lagging a bit behind my team contacting schedule and will not manage to contact every team before DebConf. As a (short) summary, I can draw some positive conclusions about my efforts to reach out to teams. I was able to identify some issues that were new to me and which I am now working on. Examples include limitations in Salsa and Salsa CI. I consider both essential parts of our infrastructure and will support both teams in enhancing their services. Some teams confirmed that they are basically using some common infrastructure (Salsa team space, mailing lists, IRC channels) but that the individual members of the team work on their own problems without sharing any common work. I have also not read about convincing strategies to attract newcomers to the team, as we have established, for instance, in the Debian Med team. DebConf attendance The amount of money needed to fly people to South Korea was higher than usual, so the DebConf bursary team had to make some difficult decisions about who could be reimbursed for travel expenses. I extended the budget for diversity and newcomers, which enabled us to invite some additional contributors. We hope that those who were not able to come this year can make it next year to Brest or to MiniDebConf Cambridge or Toulouse tag2upload On June 12, Sean Whitton requested comments on the debian-vote list regarding a General Resolution (GR) about tag2upload. The discussion began with technical details but unfortunately, as often happens in long threads, it drifted into abrasive language, prompting the community team to address the behavior of an opponent of the GR supporters. After 560 emails covering technical details, including a detailed security review by Russ Allbery, Sean finally proposed the GR on June 27, 2024 (two weeks after requesting comments). Firstly, I would like to thank the drivers of this GR and acknowledge the technical work behind it, including the security review. I am positively convinced that Debian can benefit from modernizing its infrastructure, particularly through stronger integration of Git into packaging workflows. Sam Hartman provided some historical context [1], [2], [3], [4], noting that this discussion originally took place five years ago with no results from several similarly lengthy threads. My favorite summary of the entire thread was given by Gregor Herrmann, which reflects the same gut feeling I have and highlights a structural problem within Debian that hinders technical changes. Addressing this issue is definitely a matter for the Debian Project Leader, and I will try to address it during my term. At the time of writing these bits, a proposal from ftpmaster, which is being continuously discussed, might lead to a solution. I was also asked to extend the GR discussion periods which I will do in separate mail. Talk: Debian GNU/Linux for Scientific Research I was invited to have a talk in the Systems-Facing Track of University of British Columbia (who is sponsoring rack space for several Debian servers). I admit it felt a bit strange to me after working more than 20 years for establishing Debian in scientific environments to be invited to such a talk "because I'm DPL". Kind regards Andreas.

Colin Watson: Free software activity in June 2024

My Debian contributions this month were all sponsored by Freexian. You can support my work directly via Liberapay.

Ben Hutchings: FOSS activity in June 2024

Ben Hutchings: FOSS activity in May 2024

1 July 2024

Ben Hutchings: FOSS activity in April 2024

Paul Wise: FLOSS Activities June 2024

Focus This month I didn't have any particular focus. I just worked on issues in my info bubble.

Changes

Issues

Communication
  • Respond to queries from Debian users and contributors on IRC

Sponsors All work was done on a volunteer basis.

Russell Coker: VoLTE in Australia

Introduction In Australia the 3G mobile frequencies are to be reused so they are in the process of shutting down the 3G service. That means that everyone has to use VoLTE (Voice Over LTE) for phone calls (including emergency calls). The shutdown time varies by telco, Kogan Mobile (one of the better services which has good value for money and generally works well) shut down their 3G service in January. Aldi Mobile (another one of the good services which is slightly more expensive but has included free calls to most first-world countries and uses the largest phone network) will shut theirs down at the end of August. For background there s a Fosdem talk about OpenSIPS with VoLTE and VoNR [1], it s more complex than you want to know. Also VoNR (Voice over New Radio) is the standard for 5G voice and it s different from VoLTE and has a fallback to VoLTE. Another good lecture for background information is the Fosdem talk on VoLTE at the handset end [2]. The PinePhonePro In October 2023 I tried using my PinePhonePro as my main phone but that only lasted a few days due to problems with calls and poor battery life [3]. Since then I went back to the Huawei Mate 10 Pro that I bought refurbished in June 2019 for $389. So that has been my main phone for 5 years now, giving a cost of $1.50 per week. I had tried using a Huawei Nova 7i running Android without Google Play as an experiment but that had failed, I do many things that need Android apps [4]. I followed the PinePhone wiki to get my PinePhonePro working with VoLTE [5]. That worked fine for me, the only difference from the instructions is that I had to use device /dev/ttyUSB3 and that the modem kept resetting itself during the process and when that happened I had to kill minicom and start again. After changing the setting and saving it the PinePhonePro seemed to work well with VoLTE on a Kogan Mobile SIM (so definitely not using 3G). One issue I have found is that Plasma Mobile (my preferred FOSS phone GUI) appears to have a library issue that results in polling every 14ms even when the screen is locked [6]. If you have a few processes doing that (which means the most lightly used Plasma system) it really hurts battery use. The maintainer has quite reasonably deferred action on this bug report given the KDE 6 transition. Later on in the Trixie development cycle I hope to get this issue resolved, I don t expect it to suddenly make battery life good. But it might make battery life acceptable. I am now idly considering carrying around my PinePhonePro in a powered off state for situations where I might need to do high security operations (root logins to servers or online banking) but for which carrying a laptop isn t convenient. It will do well for the turn on, do 30 mins of work that needs security, and then turn off scenario. Huawei Mate 10 Pro and Redmi 9A The Huawei Mate 10 Pro has been my main phone for 5 years and it has worked well, so it would be ideal if it could do VoLTE as the PinePhonePro isn t ready yet. All the web pages I ve seen about the Mate 10 Pro say that it will either allow upgrading to a VoLTE configuration if run with the right SIM or only support it with the right SIM. I did a test with a Chinese SIM which gave an option of turning on VoLTE but didn t allow any firmware updates and the VoLTE option went away when I put an Australian SIM in. Some forum comments had led me to believe that it would either permanently enable VoLTE or allow upgrading the firmware to one that enables VoLTE if I used a Chinese SIM but that is not the case. I didn t expect a high probability of success but I had to give it a go as it s a nice phone. I did some tests on a Redmi 9A (a terrible phone that has really bad latency on the UI in spite of having reasonably good hardware). The one I tested on didn t have VoLTE enabled when I got it, to test that I used the code *#*#4636#*#* in the dialler to get the menu of SIM information and it showed that VoLTE was not provisioned. I then had to update to the latest release of Android for that phone and enter *#*#86583#*#* in the dialler to enable VoLTE, the message displayed after entering that magic number must end in DISABLE . I get the impression that the code in question makes the phone not check certain aspects of whether the carrier is good for VoLTE and just do it. So apparently Kogan Mobile somehow gives the Redmi 9A the impression that VoLTE isn t supported but if the phone just goes ahead and connects it will work. I don t plan to use a Redmi 9A myself as it s too slow, but I added it to my collection to offer to anyone else I know who needs a phone with VoLTE and doesn t use the phone seriously or to someone who needs a known good phone for testing things. Samsung Galaxy Note 9 I got some Samsung Galaxy Note 9 phones to run Droidian as an experiment [7]. But Droidian dropped support for the Note 9 and I couldn t figure out how to enable VoLTE via Droidian, which was very annoying after I had spent $109 on a test phone and $215 on a phone for real use (I have no plans to try Droidian again at this time). I tried installing LineageOS on one Note 9 [8] which was much easier than expected (especially after previously installing Droidian). But VoLTE wasn t an option. According to Reddit LineageOS doesn t support VoLTE on Samsung devices and you can use a magisk module or a VoLTE enabler module but those aren t supported by LineageOS either [9]. I downloaded an original image for the Note 9 from SamsMobile.com [10]. That image booted past the orange stage (where if you have problems then your phone is probably permanently useless) but didn t boot into the OS. A friend helped me out with that and it turned out that the Heimdal flash tool on Linux didn t do something it needed to do and that Odin on Windows was required. After using Odin everything was fine and I have a Note 9 with VoLTE running the latest Samsung firmware which is security patch level 1st July 2022!!! So I have a choice between using a Note 9 for data and SMS while running a current version of Lineage OS with all security fixes or running a Samsung image with no security updates for 2 years which supports phone calls. So based on this I have to recommend Pixel as the phone of choice, it has a decent level of support from Google and long term support from LineageOS. According to the LineageOS web site you can run the current version of Lineage on the original Pixel phone from 2016! Of course getting VoLTE to work on it might be another saga, but it would probably be easier to do with LineageOS on a Pixel than on a Samsung phone. Conclusion The operation of the Note 9 for me is decent now apart from the potential security issues. The same goes for selling one of the phones. The PinePhonePro still has potential to become my daily driver at some future time if I and others can optimise power use. Also a complicating factor is that I want to have both Jabber and Matrix be actually instant IM systems not IM with a 5 minute delay, so suspend mode isn t a good option. Pixel phones will be a much higher priority when looking at phones to buy in future. The older Pixel phones go for as little as $100 on eBay and can still run the latest LineageOS. VoLTE seems needlessly complicated.

Niels Thykier: Debian packaging with style black

When I started working on the language server for debputy, one of several reasons was about automatic applying a formatting style. Such that you would not have to remember to manually reformat the file. One of the problems with supporting automatic formatting is that no one agrees on the "one true style". To make this concrete, Johannes Schauer Marin Rodrigues did the numbers of which wrap-and-sort option that are most common in https://bugs.debian.org/895570#46. Unsurprising, we end up with 14-15 different styles with various degrees of popularity. To make matters worse, wrap-and-sort does not provide a way to declare "this package uses options -sat". So that begged the question, how would debputy know which style it should use when it was going to reformat file. After a couple of false-starts, Christian Hofstaedtler mentioned that we could just have a field in debian/control for supporting a "per-package" setting in responds to my concern about adding a new "per-package" config file. At first, I was not happy with it, because how would you specify all of these options in a field (in a decent manner)? But then I realized that one I do not want all these styles and that I could start simpler. The Python code formatter black is quite successful despite not having a lot of personalized style options. In fact, black makes a statement out of not allowing a lot of different styles. Combing that, the result was X-Style: black (to be added to the Source stanza of debian/control), which every possible reference to the black tool for how styling would work. Namely, you outsource the style management to the tool (debputy) and then start using your focus on something else than discussing styles. As with black, this packaging formatting style is going to be opinionated and it will evolve over time. At the starting point, it is similar to wrap-and-sort -sat for the deb822 files (debputy does not reformat other files at the moment). But as mentioned, it will likely evolve and possible diverge from wrap-and-sort over time. The choice of the starting point was based on the numbers posted by Johannes #895570. It was not my personal favorite but it seemed to have a majority and is also close to the one suggested by salsa pipeline maintainers. The delta being -kb which I had originally but removed in 0.1.34 at request of Otto Kek l inen after reviewing the numbers from Johannes one more time. To facilitate this new change, I uploaded debputy/0.1.30 (a while back) to Debian unstable with the following changes:
  • Support for the X-Style: black header.
  • When a style is defined, the debputy lsp server command will now automatically reformat deb822 files on save (if the editor supports it) or on explicit "reformat file" request from the editor (usually indirectly from the user).
  • New subcommand debputy reformat command that will reformat the files, when a style is defined.
  • A new pre-commit hook repo to run debputy lint and debputy reformat. These hooks are available from https://salsa.debian.org/debian/debputy-pre-commit-hooks version v0.1 and can be used with the pre-commit tool (from the package of same name).
The obvious omission is a salsa-pipeline feature for this. Otto has put that on to his personal todo list and I am looking forward to that.
Beyond black Another thing I dislike about our existing style tooling is that if you run wrap-and-sort without any arguments, you have a higher probability of "trashing" the style of the current package than getting the desired result. Part of this is because wrap-and-sort's defaults are out of sync with the usage (which is basically what https://bugs.debian.org/895570 is about). But I see another problem. The wrap-and-sort tool explicitly defined options to tweak the style but provided maintainers no way to record their preference in any machine readable way. The net result is that we have tons of diverging styles and that you (as a user of wrap-and-sort) have to manually tell wrap-and-sort which style you want every time you run the tool. In my opinion that is not playing to the strengths of neither human nor machine. Rather, it is playing to the weaknesses of the human if anything at all. But the salsa-CI pipeline people also ran into this issue and decided to work around this deficiency. To use wrap-and-sort in the salsa-CI pipeline, you have to set a variable to activate the job and another variable with the actual options you want. The salsa-CI pipeline is quite machine readable and wrap-and-sort is widely used. I had debputy reformat also check for the salsa-CI variables as a fallback. This fallback also works for the editor mode (debputy lsp server), so you might not even have to run debputy reformat. :) This was a deliberate trade-off. While I do not want all us to have all these options, I also want Debian packaging to be less painful and have fewer paper cuts. Having debputy go extra lengths to meet wrap-and-sort users where they are came out as the better solution for me. A nice side-effect of this trade-off is that debputy reformat now a good tool for drive-by contributors. You can safely run debputy reformat on any package and either it will apply the styling or it will back out and inform you that no obvious style was detected. In the latter case, you would have to fallback to manually deducing the style and applying it.
Differences to wrap-and-sort The debputy reformat has some limitations or known differences to wrap-and-sort. Notably, debputy reformat (nor debputy lsp server) will not invoke wrap-and-sort. Instead, debputy has its own reformatting engine that provides similar features. One reason for not running wrap-and-sort is that I want debputy reformat to match the style that debputy lsp server will give you. That way, you get consistent style across all debputy commands. Another reason is that it is important to me that reformatting is safe and does not change semantics. This leads to two regrettable known differences to the wrap-and-sort behavior due to safety in addition to one scope limitation in debputy:
  1. debputy will ignore requests to sort the stanzas when the "keep first" option is disabled (-b --no-keep-first). This combination is unsafe reformatting. I feel it was a mistake for wrap-and-sort to ever allow this but at least it is no longer the default (-b is now -bk by default). This will be less of a problem in debhelper-compat 15, since the concept of "main package" will disappear and all multi-binary source packages will be required to use debian/package.install rather than debian/install.
  2. debputy will not reorder the contents of debhelper packaging files such as debian/install. This is also an (theoretical) unsafe thing to do. While the average package will not experience issues with this, there are rare corner cases where the re-ordering can affect the end result. I happen to know this, because I ran into issues when trying to optimize dh_install in a way that assumed the order did not matter. Stuff broke and there is now special-case code in dh_install to back out of that optimization when that happens.
  3. debputy has a limited list of wrap-and-sort options it understands. Some options may cause debputy to back out and disable reformatting entirely with a remark that it cannot apply that style. If you run into a case of this, feel free to file a feature request to support it. I will not promise to support everything, but if it is safe and trivially doable with the engine already, then I probably will.
As stated, where debputy cannot implement the wrap-and-sort styles fully, then it will currently implement a subset that is safe if that can be identified or back out entirely of the formatting when it cannot. In all cases, debputy will not break the formatting if it is correct. It may just fail at correcting one aspect of the wrap-and-sort style if you happen to get it wrong. It is also important to remember that the prerequisite for debputy applying any wrap-and-sort style is that you have set the salsa-CI pipeline variables to trigger wrap-and-sort with the salsa-CI pipeline. So there is still a CI check before the merge that will run the wrap-and-sort in its full glory that provides the final safety net for you.
Just give me a style In conclusion, if you, like me, are more interested in getting a consistent style rather than discussing what that style should be, now you can get that with X-Style: black. You can also have your custom wrap-and-sort style be picked up automatically for drive-by contributors.
$ apt satisfy 'dh-debputy (>= 0.1.30), python3-lsprotocol'
# Add  X-Style: black  to  debian/control  for "just give me a style"
#
# OR, if there is a specific  wrap-and-sort  style for you then set
# SALSA_CI_DISABLE_WRAP_AND_SORT=no plus set relevant options in
# SALSA_CI_WRAP_AND_SORT_ARGS in debian/salsa-ci.yml (or .gitlab-ci.yml)
$ debputy reformat
It is sadly not yet in the salsa-ci pipeline. Otto is looking into that and hopefully we will have it soon. :) And if you find yourself often doing archive-wide contributions and is tired of having to reverse engineer package formatting styles, consider using debputy reformat or debputy lsp server. If you use debputy in this way, please consider providing feedback on what would help you.

29 June 2024

Russ Allbery: Review: Bitter Angels

Review: Bitter Angels, by C.L. Anderson
Publisher: Ballantine Books
Copyright: 2009
ISBN: 0-553-59217-3
Format: Mass market
Pages: 438
Bitter Angels is a stand-alone science fiction novel. It won the Philip K. Dick award for best SF original paperback in 2010. C.L. Anderson is a pen name for Sarah Zettel. Terese was a Guardian, one of the agents of the Pax Solaris who find ways to keep the peace in troubled systems and high-stress situations with the help of an implanted Companion, an assistant AI. Forty years ago, on one of those missions, she was captured and her Companion was forcibly removed. She was rescued by her friend and mentor and retired afterwards, starting a new life and a new family, trying to leave the memories behind. Now, the woman who rescued her is dead. She was murdered on duty in the Erasmus system, a corporate hellhole that appears to be on the verge of exploding into a political hot spot. Bianca's last instructions asked for Terese to replace her. Terese's family is furious at her for even considering returning to the Guardians, but she can't say no. Duty, and Bianca's dying request, call too strongly. Amerand is Security on Dazzle, one of the Erasmus stations. He is one of the refugees from Oblivion, the station that the First Bloods who rule the system let die. He keeps his head above water and tries to protect his father and find his mother without doing anything that the ever-present Clerks might find concerning. Keeping an eye on newly-arriving Solaris saints is a typical assignment, since the First Bloods don't trust the meddling do-gooders. But something is not quite right, and a cryptic warning from his Clerk makes him even more suspicious. This is the second book by Sarah Zettel that I've read, and both of them have been tense, claustrophobic thrillers set in a world with harsh social inequality and little space for the characters to maneuver. In this case, the structure of her future universe reminded me a bit of Iain M. Banks's Culture, but with less advanced technology and only humans. The Pax Solaris has eliminated war within its borders and greatly extended lifespans. That peace is maintained by Guardians, who play a role similar to Special Circumstances but a bit more idealist and less lethal. They show up where there are problems and meddle, manipulating and pushing to try to defuse the problems before they reach the Pax Solaris. Like a Culture novel, nearly all of the action takes place outside the Pax Solaris in the Erasmus system. Erasmus is a corporate colony that has turned into a cross between a hereditary dictatorship and the Corporate Rim from Martha Wells's Murderbot series. Debt slavery is ubiquitous, economic inequality is inconceivably vast, and the Clerks are everywhere. Erasmus natives like Amerand have very little leeway and even fewer options. Survival is a matter of not drawing the attention of the wrong people. Terese and her fellow Guardians are appalled, but also keenly aware that destabilizing the local politics may make the situation even worse and get a lot of people killed. Bitter Angels is structured like a mystery: who killed Bianca, and what was her plan when she was killed? Unlike a lot of books of this type, the villains are not idiots and their plan is both satisfyingly complex and still depressingly relevant. I don't think I'm giving anything away by saying that I have read recent news articles about people with very similar plans, albeit involving less science-fiction technology. Anderson starts with a tense situation and increases the pressure relentlessly, leaving the heroes one step behind the villains for almost the entire novel. It is not happy or optimistic reading at times, the book is quite dark but it certainly was engrossing. The one world-building quibble that I had is that the Erasmus system is portrayed partly as a hydraulic empire, and while this is arguably feasible given that spaceship travel is strictly controlled, it seemed like a weird choice given the prevalence of water on the nearby moons. Water smuggling plays a significant role in the plot, and I wasn't entirely convinced of the politics and logistics behind it. If this sort of thing bugs you, there are some pieces that may require suspension of disbelief. Bitter Angels is the sort of tense thriller where catastrophe is barely avoided and the cost of victory is too high, so you will want to be in the mood for that before you dive in. But if that's what you're looking for, I thought Anderson delivered a complex and satisfying story. Content warning: major character suicide. Rating: 7 out of 10

14 June 2024

Paul Tagliamonte: Reverse Engineering a Restaurant Pager system

It s been a while since I played with something new been stuck in a bit of a rut with radios recently - working on refining and debugging stuff I mostly understand for the time being. The other day, I was out getting some food and I idly wondered how the restaurant pager system worked. Idle curiosity gave way to the realization that I, in fact, likely had the means and ability to answer this question, so I bought the first set of the most popular looking restaurant pagers I could find on eBay, figuring it d be a fun multi-week adventure.

Order up! I wound up buying a Retekess brand TD-158 Restaurant Pager System (they looked like ones I d seen before and seemed to be low-cost and popular), and quickly after, had a pack of 10 pagers and a base station in-hand. The manual stated that the radios operated at 433 MHz (cool! can do! Love a good ISM band device), and after taking an inital read through the manual for tips on the PHY, I picked out a few interesting things. First is that the base station ID was limited to 0-999, which is weird because it means the limiting factor is likely the base-10 display on the base station, not the protocol we need enough bits to store 999 at least 10 bits. Nothing else seemed to catch my eye, so I figured may as well jump right to it. Not being the type to mess with success, I did exactly the same thing as I did in my christmas tree post, and took a capture at 433.92MHz since it was in the middle of the band, and immediately got deja-vu. Not only was the signal at 433.92MHz, but throwing the packet into inspectrum gave me the identical plot of the OOK encoding scheme. Not just similar identical. The only major difference was the baud rate and bit structure of the packets, and the only minor difference was the existence of what I think is a wakeup preamble packet (of all zeros), rather than a preamble symbol that lasted longer than usual PHY symbol (which makes this pager system a bit easier to work with than my tree, IMHO). Getting down to work, I took some measurements to determine what the symbol duration was over the course of a few packets, I was able to determine the symbol rate was somewhere around 858 microseconds (0.000858 seconds per symbol), which is a weird number, but maybe I m slightly off or there s some larger math I m missing that makes this number satisfyingly round (internal low cost crystal clock or something? I assume this is some hardware constrait with the pager?) Anyway, good enough. Moving along, let s try our hand at a demod let s just assume it s all the same as the chrismas tree post and demod ones and zeros the same way here. That gives us 26 bits:
00001101110000001010001000
Now, I know we need at least 10 bits for the base station ID, some number of bits for the pager ID, and some bits for the command. This was a capture of me hitting call from a base station ID of 55 to a pager with the ID of 10, so let s blindly look for 10 bit chunks with the numbers we re looking for:
0000110111 0000001010 001000
Jeez. First try. 10 bits for the base station ID (55 in binary is 0000110111), 10 bits for the pager ID (10 in binary is 0000001010), which leaves us with 6 bits for a command (and maybe something else too?) which is 8 here. Great, cool, let s work off that being the case and revisit it if we hit bugs. Besides our data packet, there s also a preamble packet that I ll add in, in case it s used for signal detection or wakeup or something which is fairly easy to do since it s the same packet structure as the above, just all zeros. Very kind of them to leave it with the same number of bits and encoding scheme it s nice that it can live outside the PHY. Once I got here, I wrote a quick and dirty modulator, and was able to ring up pagers! Unmitigated success and good news only downside was that it took me a single night, and not the multi-week adventure I was looking for. Well, let s finish the job and document what we ve found for the sake of completeness.

Boxing everything up My best guess on the packet structure is as follows:
base id
argument
command
For a call or F2 operation, the argument is the Pager s ID code, but for other commands it s a value or an enum, depending. Here s a table of my by-hand demodulation of all the packet types the base station produces:
Type Cmd Id Description
Call8Call the pager identified by the id in argument
Off60Request any pagers on the charger power off when power is removed, argument is all zero
F240Program a pager to the specified Pager ID (in argument) and base station
F344Set the reminder duration in seconds specified in argument
F448Set the pager's beep mode to the one in argument (0 is disabled, 1 is slow, 2 is medium, 3 is fast)
F552Set the pager's vibration mode to the one in argument (0 is disabled, 1 is enabled)

Kitchen s closed for the night I m not going to be publishing this code since I can t think of a good use anyone would have for this besides folks using a low cost SDR and annoying local resturants; but there s enough here for folks who find this interesting to try modulating this protocol on their own hardware if they want to buy their own pack of pagers and give it a shot, which I do encourage! It s fun! Radios are great, and this is a good protocol to hack with it s really nice. All in all, this wasn t the multi-week adventure I was looking for, this was still a great exercise and a fun reminder that I ve come a far way from when I ve started. It felt a lot like cheating since I was able to infer a lot about the PHY because I d seen it before, but it was still a great time. I may grab a few more restaurant pagers and see if I can find one with a more exotic PHY to emulate next. I mean why not, I ve already got the thermal printer libraries working

Next.