Search Results: "Edward Betts"

7 October 2022

Edward Betts: Fish shell now has underscore as a number separator (my feature request)

In November 2021 I filed a feature request for the fish shell to add underscore as a thousand separator in numbers. My feature request has been implemented and is available in fish 3.5.0, released 16 June 2022. The fish shell supports mathematical operations using the math command. 
edward@x1c9 ~> math 2_000 + 22
2022
edward@x1c9 ~>
The underscore can be used as a thousand separator, but there are other uses for a number separator. Here's a list taken from a post by Mathias Bynens about the number separator in JavaScript: 
// A decimal integer literal with its digits grouped per thousand:
1_000_000_000_000
// A decimal literal with its digits grouped per thousand:
1_000_000.220_720
// A binary integer literal with its bits grouped per octet:
0b01010110_00111000
// A binary integer literal with its bits grouped per nibble:
0b0101_0110_0011_1000
// A hexadecimal integer literal with its digits grouped by byte:
0x40_76_38_6A_73
// A BigInt literal with its digits grouped per thousand:
4_642_473_943_484_686_707n
Programming languages are gradually adding a number separator to their syntax, I think Perl was the first. Most are languages use underscore, but C++ 14 uses an apostrophe for the number separator.

13 July 2022

Reproducible Builds: Reproducible Builds in June 2022

Welcome to the June 2022 report from the Reproducible Builds project. In these reports, we outline the most important things that we have been up to over the past month. As a quick recap, whilst anyone may inspect the source code of free software for malicious flaws, almost all software is distributed to end users as pre-compiled binaries.

Save the date! Despite several delays, we are pleased to announce dates for our in-person summit this year: November 1st 2022 November 3rd 2022
The event will happen in/around Venice (Italy), and we intend to pick a venue reachable via the train station and an international airport. However, the precise venue will depend on the number of attendees. Please see the announcement mail from Mattia Rizzolo, and do keep an eye on the mailing list for further announcements as it will hopefully include registration instructions.

News David Wheeler filed an issue against the Rust programming language to report that builds are not reproducible because full path to the source code is in the panic and debug strings . Luckily, as one of the responses mentions: the --remap-path-prefix solves this problem and has been used to great effect in build systems that rely on reproducibility (Bazel, Nix) to work at all and that there are efforts to teach cargo about it here .
The Python Security team announced that:
The ctx hosted project on PyPI was taken over via user account compromise and replaced with a malicious project which contained runtime code which collected the content of os.environ.items() when instantiating Ctx objects. The captured environment variables were sent as a base64 encoded query parameter to a Heroku application [ ]
As their announcement later goes onto state, version-pinning using hash-checking mode can prevent this attack, although this does depend on specific installations using this mode, rather than a prevention that can be applied systematically.
Developer vanitasvitae published an interesting and entertaining blog post detailing the blow-by-blow steps of debugging a reproducibility issue in PGPainless, a library which aims to make using OpenPGP in Java projects as simple as possible . Whilst their in-depth research into the internals of the .jar may have been unnecessary given that diffoscope would have identified the, it must be said that there is something to be said with occasionally delving into seemingly low-level details, as well describing any debugging process. Indeed, as vanitasvitae writes:
Yes, this would have spared me from 3h of debugging But I probably would also not have gone onto this little dive into the JAR/ZIP format, so in the end I m not mad.

Kees Cook published a short and practical blog post detailing how he uses reproducibility properties to aid work to replace one-element arrays in the Linux kernel. Kees approach is based on the principle that if a (small) proposed change is considered equivalent by the compiler, then the generated output will be identical but only if no other arbitrary or unrelated changes are introduced. Kees mentions the fantastic diffoscope tool, as well as various kernel-specific build options (eg. KBUILD_BUILD_TIMESTAMP) in order to prepare my build with the known to disrupt code layout options disabled .
Stefano Zacchiroli gave a presentation at GDR S curit Informatique based in part on a paper co-written with Chris Lamb titled Increasing the Integrity of Software Supply Chains. (Tweet)

Debian In Debian in this month, 28 reviews of Debian packages were added, 35 were updated and 27 were removed this month adding to our knowledge about identified issues. Two issue types were added: nondeterministic_checksum_generated_by_coq and nondetermistic_js_output_from_webpack. After Holger Levsen found hundreds of packages in the bookworm distribution that lack .buildinfo files, he uploaded 404 source packages to the archive (with no meaningful source changes). Currently bookworm now shows only 8 packages without .buildinfo files, and those 8 are fixed in unstable and should migrate shortly. By contrast, Debian unstable will always have packages without .buildinfo files, as this is how they come through the NEW queue. However, as these packages were not built on the official build servers (ie. they were uploaded by the maintainer) they will never migrate to Debian testing. In the future, therefore, testing should never have packages without .buildinfo files again. Roland Clobus posted yet another in-depth status report about his progress making the Debian Live images build reproducibly to our mailing list. In this update, Roland mentions that all major desktops build reproducibly with bullseye, bookworm and sid but also goes on to outline the progress made with automated testing of the generated images using openQA.

GNU Guix Vagrant Cascadian made a significant number of contributions to GNU Guix: Elsewhere in GNU Guix, Ludovic Court s published a paper in the journal The Art, Science, and Engineering of Programming called Building a Secure Software Supply Chain with GNU Guix:
This paper focuses on one research question: how can [Guix]((https://www.gnu.org/software/guix/) and similar systems allow users to securely update their software? [ ] Our main contribution is a model and tool to authenticate new Git revisions. We further show how, building on Git semantics, we build protections against downgrade attacks and related threats. We explain implementation choices. This work has been deployed in production two years ago, giving us insight on its actual use at scale every day. The Git checkout authentication at its core is applicable beyond the specific use case of Guix, and we think it could benefit to developer teams that use Git.
A full PDF of the text is available.

openSUSE In the world of openSUSE, SUSE announced at SUSECon that they are preparing to meet SLSA level 4. (SLSA (Supply chain Levels for Software Artifacts) is a new industry-led standardisation effort that aims to protect the integrity of the software supply chain.) However, at the time of writing, timestamps within RPM archives are not normalised, so bit-for-bit identical reproducible builds are not possible. Some in-toto provenance files published for SUSE s SLE-15-SP4 as one result of the SLSA level 4 effort. Old binaries are not rebuilt, so only new builds (e.g. maintenance updates) have this metadata added. Lastly, Bernhard M. Wiedemann posted his usual monthly openSUSE reproducible builds status report.

diffoscope diffoscope is our in-depth and content-aware diff utility. Not only can it locate and diagnose reproducibility issues, it can provide human-readable diffs from many kinds of binary formats. This month, Chris Lamb prepared and uploaded versions 215, 216 and 217 to Debian unstable. Chris Lamb also made the following changes:
  • New features:
    • Print profile output if we were called with --profile and we were killed via a TERM signal. This should help in situations where diffoscope is terminated due to some sort of timeout. [ ]
    • Support both PyPDF 1.x and 2.x. [ ]
  • Bug fixes:
    • Also catch IndexError exceptions (in addition to ValueError) when parsing .pyc files. (#1012258)
    • Correct the logic for supporting different versions of the argcomplete module. [ ]
  • Output improvements:
    • Don t leak the (likely-temporary) pathname when comparing PDF documents. [ ]
  • Logging improvements:
    • Update test fixtures for GNU readelf 2.38 (now in Debian unstable). [ ][ ]
    • Be more specific about the minimum required version of readelf (ie. binutils), as it appears that this patch level version change resulted in a change of output, not the minor version. [ ]
    • Use our @skip_unless_tool_is_at_least decorator (NB. at_least) over @skip_if_tool_version_is (NB. is) to fix tests under Debian stable. [ ]
    • Emit a warning if/when we are handling a UNIX TERM signal. [ ]
  • Codebase improvements:
    • Clarify in what situations the main finally block gets called with respect to TERM signal handling. [ ]
    • Clarify control flow in the diffoscope.profiling module. [ ]
    • Correctly package the scripts/ directory. [ ]
In addition, Edward Betts updated a broken link to the RSS on the diffoscope homepage and Vagrant Cascadian updated the diffoscope package in GNU Guix [ ][ ][ ].

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

Testing framework The Reproducible Builds project runs a significant testing framework at tests.reproducible-builds.org, to check packages and other artifacts for reproducibility. This month, the following changes were made:
  • Holger Levsen:
    • Add a package set for packages that use the R programming language [ ] as well as one for Rust [ ].
    • Improve package set matching for Python [ ] and font-related [ ] packages.
    • Install the lz4, lzop and xz-utils packages on all nodes in order to detect running kernels. [ ]
    • Improve the cleanup mechanisms when testing the reproducibility of Debian Live images. [ ][ ]
    • In the automated node health checks, deprioritise the generic kernel warning . [ ]
  • Roland Clobus (Debian Live image reproducibility):
    • Add various maintenance jobs to the Jenkins view. [ ]
    • Cleanup old workspaces after 24 hours. [ ]
    • Cleanup temporary workspace and resulting directories. [ ]
    • Implement a number of fixes and improvements around publishing files. [ ][ ][ ]
    • Don t attempt to preserve the file timestamps when copying artifacts. [ ]
And finally, node maintenance was also performed by Mattia Rizzolo [ ].

Mailing list and website On our mailing list this month: Lastly, Chris Lamb updated the main Reproducible Builds website and documentation in a number of small ways, but primarily published an interview with Hans-Christoph Steiner of the F-Droid project. Chris Lamb also added a Coffeescript example for parsing and using the SOURCE_DATE_EPOCH environment variable [ ]. In addition, Sebastian Crane very-helpfully updated the screenshot of salsa.debian.org s request access button on the How to join the Salsa group. [ ]

Contact If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

15 June 2022

Edward Betts: Find link needs a rewrite, the visual editor broke it

Find link is a tool that I wrote for adding links between articles in Wikipedia. Given an article title, find link will find other articles that include the entered article title but no link to the article. There is the option to edit the found articles and add the missing link. For example, you might want to find missing links to the gig economy article.
I originally wrote the tool in 2008 when the MediaWiki software didn't have a rich-text editor. Wikipedia articles were edited by writing wiki markup in MediaWiki syntax. Since then MediaWiki has evolved and now has rich-text editing via the visual editor. Users don't need to know how to write wiki markup to modify an article. Within MediaWiki there is a user preference to disable the visual editor and stick with editing via the original wiki markup. Find link edits articles by taking the article text, adding the missing link, and sending the user to the changes view of the modified article on Wikipedia, if they're happy with the change they hit save. This only works with the original editor, it doesn't work with the visual editor.
English Wikipedia has had the visual editor enabled by default since 2016. For somebody to use find link they need to disable the visual editor in their Wikipedia preferences first. Fixing this bug means quite a significant change to how the tool works. My plan is to rewrite find link to save edits directly without needing to send the user to Wikipedia article edit change view page to make the edits. Users will authenticate with their Wikipedia account via OAuth and give permission for find link to edit articles on their behalf. Some of my other tools use OAuth for editing OpenStreetMap and Wikidata, so I'm confident about using it to edit Wikipedia. The source code for find link is on GitHub. I'll post updates here as I make progress on the rewrite.

13 June 2022

Edward Betts: Fixing spelling in GitHub repos using codespell

Codespell is a spell checker specifically designed for finding misspellings in source code. I've been using it to correct spelling mistakes in GitHub repos sine 2016. Most spell checkers use a list of valid words and highlighting any word in a document that is not in the word list. This method doesn't work for source code because code contains abbreviations and words joined together without spaces, a spell checker will generate too many false positives. Codespell uses a different approach, instead of a list of valid words it has a dictionary of common misspellings. Currently the codespell dictionary includes 34,466 known misspellings. I've contributed 300 misspellings to the dictionary. Whenever I find an interesting open source project I run codespell to check for spelling mistakes. Most projects have spelling mistakes and I can send a pull request to fix them. In 2019 Microsoft made the Windows calculator open source and uploaded it to GitHub. I used codespell to find some spelling mistakes, sent them a pull request and they accepted it. A great source for GitHub repos to spell check is Hacker News. Let's have a look.
Hacker News has a link to forum software called Flarum. I can use codespell to look for spelling mistakes. When I'm looking for errors in a GitHub repo I don't fork the project until I know there is a spelling mistake to fix. 
edward@x1c9 ~/spelling> git clone git@github.com:flarum/flarum.git
Cloning into &aposflarum&apos...
remote: Enumerating objects: 1338, done.
remote: Counting objects: 100% (42/42), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 1338 (delta 21), reused 36 (delta 19), pack-reused 1296
Receiving objects: 100% (1338/1338), 725.02 KiB   1.09 MiB/s, done.
Resolving deltas: 100% (720/720), done.
edward@x1c9 ~/spelling> cd flarum/
edward@x1c9 ~/spelling/flarum (master)> codespell -q3
./public/web.config:13: sensitve ==> sensitive
edward@x1c9 ~/spelling/flarum (master)> gh repo fork
  Created fork EdwardBetts/flarum
? Would you like to add a remote for the fork? Yes
  Added remote origin
edward@x1c9 ~/spelling/flarum (master)> git checkout -b spelling
Switched to a new branch &aposspelling&apos
edward@x1c9 ~/spelling/flarum (spelling)> codespell -q3
./public/web.config:13: sensitve ==> sensitive
edward@x1c9 ~/spelling/flarum (spelling)> codespell -q3 -w
FIXED: ./public/web.config
edward@x1c9 ~/spelling/flarum (spelling)> git commit -am "Correct spelling mistakes"
[spelling bbb04c7] Correct spelling mistakes
 1 file changed, 1 insertion(+), 1 deletion(-)
edward@x1c9 ~/spelling/flarum (spelling)> git push -u origin
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 8 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 360 bytes   360.00 KiB/s, done.
Total 4 (delta 3), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
remote: 
remote: Create a pull request for &aposspelling&apos on GitHub by visiting:
remote:      https://github.com/EdwardBetts/flarum/pull/new/spelling
remote: 
To github.com:EdwardBetts/flarum.git
 * [new branch]      spelling -> spelling
branch &aposspelling&apos set up to track &aposorigin/spelling&apos.
edward@x1c9 ~/spelling/flarum (spelling)> gh pr create 
Creating pull request for EdwardBetts:spelling into master in flarum/flarum
? Title Correct spelling mistakes
? Choose a template Open a blank pull request
? Body <Received>
? What&aposs next? Submit
https://github.com/flarum/flarum/pull/81
edward@x1c9 ~/spelling/flarum (spelling)>
That worked. I found one spelling mistake, the word "sensitive" was spelled wrong. I forked the repo, fixed the spelling mistake and submitted the fix as a pull request.
The maintainer of Flarum accepted my pull request. Fixing spelling mistakes in Bootstrap helped me unlocked the Mars 2020 Contributor achievements on GitHub.
Why not try running codespell on your own codebase? You'll probably find some spelling mistakes to fix.

31 October 2020

Jonathan Carter: Free Software Activities for 2020-10

Another month, another bunch of uploads. The freeze for Debian 11 (bullseye) is edging closer, so I ve been trying to get my package list in better shape ahead of that. Thanks to those who worked on fixing lintian.debian.org and the lintian reports on the QA pages, those are immensely useful and it s great to have that back! 2020-10-04: Upload package gnome-shell-extension-draw-on-your-screen (8-1) to Debian unstable. 2020-10-05: Sponsor package flask-restful (0.3.8-4) for Debian unstable (Python Team request). 2020-10-05: Sponsor package python-potr (1.0.2-3) for Debian unstable (Python Team request). 2020-10-06: Sponsor package python-pyld (2.0.3-1) for Debian unstable (Python Team request). 2020-10-06: Sponsor package flask-openid (1.2.5+dfsg-4) for Debian unstable (Python Team request). 2020-10-06: Sponsor package qosmic (1.6.0-4) for Debian unstable (E-mail request). 2020-10-07: File removal for gnome-shell-extension-workspace-to-dock (RC Buggy, no longer maintained: #971803). 2020-10-07: Upload package gnome-shell-extension-pixelsaver (1.20-2) to Debian unstable (Closes: #971689). 2020-10-07: Upload package calamares (3.2.31-1) to Debian unstable. 2020-10-07: Upload package gnome-shell-extension-dashtodock (69-1) to Debian unstable (Closes: #971654). 2020-10-08: Sponsor package python3-libcloud (3.020-1) for Debian unstable. 2020-10-09: Upload package gnome-shell-extension-dashtopanel (40-1) to Debian unstable (Closes: #971087). 2020-10-09: Upload package gnome-shell-extension-draw-on-your-screen (8.1-1) to Debian unstable. 2020-10-12: Upload package gnome-shell-extension-pixelsaver (1.24-1) to Debian unstable. 2020-10-14: Sponsor package python3-onewire (0.2-1) for Debian unstable (Python Team request). 2020-10-15: Sponsor package cheetah (3.2.5-1) for Debian unstable (Python Team request). 2020-10-15: Sponsor package xmodem (0.4.6+dfsg-1) for Debian unstable (Python Team request). 2020-10-15: Sponsor package ansi (0.1.5-1) for Debian unstable (Python Team request). 2020-10-15: Sponsor package cbor2 (5.2.0-1) for Debian unstable (Python Team request). 2020-10-16: Upload package calamares (3.2.32-1) to Debian unstable. 2020-10-17: Upload package calamares (3.2.32.1-1) to Debian unstable. 2020-10-18: Upload package kpmcore (4.2.0-1) to Debian unstable. 2020-10-18: Upload package gnome-shell-extension-draw-on-your-screen (9-1) to Debian unstable. 2020-10-18: Upload package bundlewrap (4.2.1-1) to Debian unstable. 2020-10-18: Upload package bcachefs-tools (0.1+git20201017.8a4408-1~exp1) to Debian experimental. 2020-10-18: Upload package calamares (3.2.32.1-2) to Debian unstable. 2020-10-18: Upload package partitionmanager (4.1.0-2) to Debian unstable. 2020-10-19: Upload package kpmcore (4.2.0-2) to Debian unstable. 2020-10-21: Upload package calamares (3.2.32.1-3) to Debian unstable. 2020-10-21: Upload package calamares-settings-debian (11.0.3-1) to Debian unstable (Closes: #969930, #941301). 2020-10-21: Upload package partitionmanager (4.2.0-1) to Debian unstable. 2020-10-21: Upload package gnome-shell-extension-hard-disk-led (22-1) to Debian unstable (Closes: #971041). 2020-10-21: Merge MR!1 for catimg (Janitor improvements). 2020-10-21: Sponsor package r4d (1.7-1) for Debian unstable (Python Team request). 2020-10-22: Upload package aalib (1.4rc5-47) to Debian unstable. 2020-10-22: Upload package fabulous (0.3.0+dfsg1-8) to Debian unstable. 2020-10-22: Merge MR!1 for gdisk (Janitor improvements). 2020-10-22: Merge MR!1 for gnome-shell-extension-arc-menu (New upstream URLs, thanks Edward Betts). 2020-10-22: Upload package gnome-shell-extension-arc-menu (49-1) to Debian unstable. 2020-10-22: Upload package gnome-shell-extension-draw-on-your-screen (10-1) to Debian unstable. 2020-10-22: Merge MR!1 for vim-airline (Janitor improvements). 2020-10-22: Merge MR!1 for vim-airline-themes (Janitor improvements). 2020-10-22: Merge MR!1 for preload (Janitor improvements). 2020-10-22: Upload package aalib (1.4rc5-48) to Debian unstable. 2020-10-22: Upload package gnome-shell-extension-trash (0.2.0-git20200326.3425fcf1-1). 2020-10-26: Upload package bcachefs-tools (0.1+git20201025.742dbbdb-1) to Debian unstable. 2020-10-26: Sponsor package dunst (1.5.0-1) for Debian unstable (mentors.debian.net request).

12 January 2016

Bits from Debian: New Debian Developers and Maintainers (November and December 2015)

The following contributors got their Debian Developer accounts in the last two months: The following contributors were added as Debian Maintainers in the last two months: Congratulations!

11 November 2015

Bits from Debian: New Debian Developers and Maintainers (September and October 2015)

The following contributors got their Debian Developer accounts in the last two months: The following contributors were added as Debian Maintainers in the last two months: Congratulations!

11 August 2006

Jonathan McDowell: So, where did you two meet?

I get asked this a lot about people. Most recently when visiting the Fat Cat with Simon and his work mates this week, but often about the other Simon (who's my business partner as well as a long time friend). The truth is a lot of the people I know I met first online, be it Fidonet (hello Pads, Peter, Simon and several others), Usenet (hello Ox.Net), mailing lists (hello ALUG) or IRC (hello, er, lots of people). I don't think I'm in any way unique here, but it can sometimes be awkward explaining this to people who preconceive internet friendships to be something seedy involving 40 year old fat man and 15 year old naive children. Or at least that's what they seem to be thinking when you say "Oh, we met online". Let's give some examples. I met Simon on Fidonet, back when we were both 17 or so. We were in a couple of echoes together and we did netmail a bit. When I came to England for university one of the echoes we were in had a meet up and so I decided to go. I ended up staying at Simon's (making his mother worry a bit; even back in 1997 people thought meeting online was freaky!) and that was the first time I met him. We've kept in good touch ever since and even gone into business together. Is this a lot odder than a chance meeting at a pub or through friends? I don't think so, but some people do. Or take when I moved to Norwich. I had some friends who could help me load up the van in Harpenden (where I was moving from), but didn't know anyone in Norwich to help unload. I'd already joined the ALUG list, so I thought I'd ask there if anyone was prepared to help in exchange for beer and food afterwards. Adam and Edward Betts both turned up to help, not knowing anything about me. And were very helpful and we got everything unloaded. I still see Adam reasonably often, both online and in person (last night, for example). I haven't really kept in touch with Edward since he left UEA though. :( There are many more examples like this of people who I see a reasonable amount in real life and yet if you asked me I'd have to admit I first knew online, sometimes for several years before actually physically meeting. And lots of people understand it these days, but please tell me I'm not alone in getting the funny looks sometimes. Please?