Russ Allbery: Review: Bookshops & Bonedust
Series: | Legends & Lattes #2 |
Publisher: | Tor |
Copyright: | 2023 |
ISBN: | 1-250-88611-2 |
Format: | Kindle |
Pages: | 337 |
Series: | Legends & Lattes #2 |
Publisher: | Tor |
Copyright: | 2023 |
ISBN: | 1-250-88611-2 |
Format: | Kindle |
Pages: | 337 |
We are glad to announce the upcoming Reproducible Builds Summit, set to take place from October 31st to November 2nd, 2023, in the vibrant city of Hamburg, Germany. This year, we are thrilled to host the seventh edition of this exciting event following the success of previous summits in various iconic locations around the world, including Venice (2022), Marrakesh (2019), Paris (2018), Berlin (2017), Berlin (2016) Athens (2015). If you re excited about joining us this year, please make sure to read the event page which has more details about the event and location. As in previous years, we will be sending invitations to all those who attended our previous summit events or expressed interest to do so. However also without receiving such a personal invitation please do email the organizers and we will find a way to accommodate you.
Series: | Seraphina #2 |
Publisher: | Ember |
Copyright: | 2015 |
ISBN: | 0-375-89659-7 |
Format: | Kindle |
Pages: | 458 |
Illustrator: | Pauline Baynes |
Series: | Chronicles of Narnia #5 |
Publisher: | Collier Books |
Copyright: | 1954 |
Printing: | 1978 |
ISBN: | 0-02-044200-9 |
Format: | Mass market |
Pages: | 217 |
Publisher: | Red Wombat Studio |
Copyright: | 2020 |
ASIN: | B0848Q8JVW |
Format: | Kindle |
Pages: | 399 |
strip-nondeterminism
and also filed #862073 against dak to upload buildinfo files to external services.misc.git:has-only.py
, and started looking at Britney.ssh
or controlling light and music with an webbrowser without authentication (besides being in the right network).
(This wasn't the hackathon per-se, but some of us appreciated these sights and so we thought you would too.)
Many thanks to:
0.033-1
and -2
were uploaded to unstable by Chris Lamb. It included contributions from:0.6.1
and 0.6.2
were uploaded to unstable by Ximin Luo. It included contributions from:Publisher: | Bantam Spectra |
Copyright: | May 1988 |
Printing: | July 1989 |
ISBN: | 0-553-27903-3 |
Format: | Mass market |
Pages: | 552 |
She shimmers, my city, she shimmers. She is said to be the most beautiful of all the cities of the Civilized Worlds, more beautiful even than Parpallaix or the cathedral cities of Vesper. To the west, pushing into the green sea like a huge, jewel-studded sleeve of city, the fragile obsidian cloisters and hospices of the Farsider's Quarter gleamed like black glass mirrors. Straight ahead as we skated, I saw the frothy churn of the Sound and their whitecaps of breakers crashing against the cliffs of North Beach and above the entire city, veined with purple and glazed with snow and ice, Waaskel and Attakel rose up like vast pyramids against the sky. Beneath the half-ring of extinct volcanoes (Urkel, I should mention, is the southernmost peak, and though less magnificent than the others, it has a conical symmetry that some find pleasing) the towers and spires of the Academy scattered the dazzling false winter light so that the whole of the Old City sparkled.That's less than half of that paragraph, and the entire book is written like that, even in the middle of conversations. Endless, constant words piled on words about absolutely everything, whether important or not, whether emotionally significant or not. And much of it isn't even description, but philosophical ponderings that are desperately trying to seem profound. Here's another bit:
Although I knew I had never seen her before, I felt as if I had known her all my life. I was instantly in love with her, not, of course, as one loves another human being, but as a wanderer might love a new ocean or a gorgeous snowy peak he has glimpsed for the first time. I was practically struck dumb by her calmness and her beauty, so I said the first stupid thing which came to mind. "Welcome to Neverness," I told her.Now, I should be fair: some people like this kind of description, or at least have more tolerance for it than I do. But that brings me to the second problem: there isn't a single truly likable character in this entire novel. Ringess, the person telling us this whole story, is a spoiled man-child, the sort of deeply immature and insecure person who attempts to compensate through bluster, impetuousness, and refusing to ever admit that he made a mistake or needed to learn something. He spends a good portion of the book, particularly the deeply bizarre and off-putting sections with the fake Neanderthals, attempting to act out some sort of stereotyped toxic masculinity and wallowing in negative emotions. Soli is an arrogant, abusive asshole from start to finish. Katherine, Ringess's love interest, is a seer who has had her eyes removed to see the future (I cannot express how disturbing I found Zindell's descriptions of this), has bizarre and weirdly sexualized reactions to the future she never explains, and leaves off the ends of all of her sentences, which might be be the most pointlessly irritating dialogue quirk I've seen in a novel. And Ringess's mother is a man-hating feminist from a separatist culture who turns into a master manipulator (I'm starting to see why Card liked this book). I at least really wanted to like Bardo, Ringess's closest friend, who has a sort of crude loyalty and unwillingness to get pulled too deep into the philosophical quicksand lurking underneath everything in this novel. Alas, Zindell insists on constantly describing Bardo's odious eating, belching, and sexual habits every time he's on the page, thus reducing him to the disgusting buffoon who gets drunk a lot and has irritating verbal ticks. About the only person I could stand by the end of the book was Justine, who at least seems vaguely sensible (and who leaves the person who abuses her), but she's too much of a non-entity to carry sustained interest. (There is potential here for a deeply scathing and vicious retelling of this story from Justine's point of view, focusing on the ways she was belittled, abused, and ignored, but I think Zindell was entirely unaware of why that would be so effective.) Oh, and there's lots of gore and horrific injury and lovingly-described torture, because of course there is. And that brings me back to the second half of that St. Louis Post-Dispatch review quote: "... really comes to life among the intrigues of Neverness." I would love to know what was hiding behind the ellipses in this pull quote, because this half-sentence is not wrong. Insofar as Neverness has any real appeal, it's in the intrigues of the city of Neverness and in the political structure that rules it. What this quote omits is that these intrigues start around page 317, more than halfway through the novel. That's about the point where faux-Wolfe starts mixing with late-career Frank Herbert and we get poet-assassins, some revelations about the leader of the Pilot culture, and some more concrete explanations of what this mess of a book is about. Unfortunately, you have to read through the huge and essentially meaningless Neanderthal scenes to get there, scenes that have essentially nothing to do with the interesting content of this book. (Everything that motivates them turns out to be completely irrelevant to the plot and useless for the characters.) The last 40% of the book is almost passable, and characters I cared about might have even made it enjoyable. Still, a couple of remaining problems detract heavily, chief among them the lack of connection of the great revelation of the story to, well, anything in the story. We learn at the very start of the novel that the stars of the Vild are mysteriously exploding, and much of the novel is driven by uncovering an explanation and solution. The characters do find an explanation, but not through any investigation. Ringess is simply told what is happening, in a wad of exposition, as a reward for something else entirely. It's weirdly disconnected from and irrelevant to everything else in the story. (There are some faint connections to the odd technological rules that the Pilot society lives under, but Zindell doesn't even draw attention to those.) The political intrigue in Neverness is similar: it appears out of nowhere more than halfway through the book, with no dramatic foundation for the motives of the person who has been keeping most of the secrets. And the final climax of the political machinations involves a bunch of mystical nonsense masquerading as science, and more of the Neanderthal bullshit that ruins the first half of the book. This is a thoroughly bad book: poorly plotted, poorly written, clotted and pretentious in style, and full of sociopaths and emotionally stunted children. I read the whole thing because I'm immensely stubborn and make poor life choices, but I was saying the eight deadly words ("I don't care what happens to these people") by a hundred pages in. Don't emulate my bad decisions. (Somehow, this novel was shortlisted for the Arthur C. Clarke award in 1990. What on earth could they possibly have been thinking?) Neverness is a stand-alone novel, but the ending sets up a subsequent trilogy that I have no intention of reading. Followed by The Broken God. Rating: 2 out of 10
"How much saffron should I add?"
"this much."
"How much is this much in SI units?"
"You're annoying me. Get out."Fast forward to March of this year. For my birthday, my wife got me a Fitbit fitness tracker. This is what I had needed all this time. It measure heart rate, distance travelled, time slept and several other pieces of info you can use to really plan a fitness regimen rationally. For example, I was chagrined to learn that sometimes when I'm at the computer, I am so immobile that the fitbit thought I was asleep. So I started planning to taken more frequent breaks. (A recent firmware upgrade has added the ability to nudge to walk atleast 250 paces each daytime hour which is handy for this.) Also by checking my heart rate I discovered that I went on the treadmill I ran too fast thereby stressing my body for little gain and ending up going too slow to get much aerobic effect. Now I can pace myself appropriately for maximum cardiac efficiency without ending up injuring myself and giving up. I also get a little more activity each day by simple changes such as taking the stairs instead of the lift and instead of getting off at the 14th street PATH I go all the way to 34th street and walk down. Tip 2: You must have data in order to see what you did right or wrong and to plan what you need to do moving forward. One caveat about these fitness trackers. They are not anywhere as accurate as a proper checkup from a doctor who specializes in such things. If you want to do any kind of pro or amateur athletics you probably should not rely on them but for the average shlub who just wants to avoid appearing on the news being winched off his sofa by the fire brigade they are good enough. Another practice I began was keeping a food diary. It can be a real eye-opener to see how much you are actually eating. It is probably much more than you thought. I am fortunate that my diet is pretty good to begin with. Vegetarian, (not vegan, Hindus eat dairy products,) mostly home-cooked with fresh ingredients, not fried or processed, and I don't drink alcohol. However there were a few optimizations I could make. I drink a lot of soda; atleast two cans a day. I really ought to stop altogether but in lieu of that I have atleast switched from Coke to Coke Zero thereby saving a lot of empty calories. I now eat 4 rotlis with my dinner instead of six. We as a family eat more green vegetables instead of potatos, skim milk instead of whole fat, canola oil instead of corn oil, and less rice and don't slather ghee on everything quite so much. One entirely new practice I've adopted that may seem faddish but works for me is intermittent fasting. The idea is to steadily train your body to need less food by eating all your days allowed amount pf calories during a 6-8 hour window and not eating at all during the remaining time. It's hard to get used to for many people but I fast atleast 2-3 times a month for religious reasons anyway so I adapted pretty quickly. The fitbit tells me how many calories I am expending and how many I can eat to maintain a healthy level of weight loss but other than that I don't bother with "food groups" or specific diets such as paleo, or low-carb etc. As long as what you eat is reasonably balanced and you are burning more calories than you are adding, it should be enough for weight loss. Indeed from the end of March to now, I've lost 3 stones (20Kg) even with the occasional "cheat" day. Tip 3: All published diets are bullshit without scientifically proven efficacy. Don't bother with them. Experiment instead and see what works for you and your metabolism. As long as you are getting all the proper nutrients (you shouldn't need a supplement unless you have an actual medical condition.) and you have a net calorie deficit, it's all good. If you eat food you enjoy, you are more likely to stick to your diet. The proper amount of sleep is one area of a healthy lifestyle I am still doing poorly in and the reasons are not all raven-related. I have always had problems with insomnia and was once actually diagnosed with sleep apnea. Losing weight has helped a lot but the fitbit is still reporting that I toss and turn a lot during the night. And that's when I'm in bed in the first place. I stay up much too late which can also lead to subsidiary bad behaviours such as midnight snacking. It's something I need to work on. Tip 4: Stop blogging at all hours of the night, It's not doing you any good. So that's what I'm doing. Moving forward, I need to deal with the sleep thing and I would also like to start some program of strength-training, I'm doing ok in terms of aerobic exercise but from what I've read, you also have to build up muscles to keep weight loss permanent. The difficulty is that it would involve joining a gym and then actually going to that gym so I've put it off for now. The immediate threat is Diwali (and Thanksgiving and Christmas...) My wife bought 4 lbs of sweets today and I can feel their presence in the fridge calling to me.
ibus-table-createdb
deterministic.
Niko Tyni wrote a patch to make libmodule-build-perl linking order deterministic.
Santiago Vila has been leading discussions on the best way to fix timestamps coming from Gettext POT files.
Packages fixed
The following 35 packages became reproducible due to changes in their
build dependencies:
apache-log4j2,
dctrl-tools,
dms,
gitit,
gnubik,
isrcsubmit,
mailutils,
normaliz,
oaklisp,
octave-fpl,
octave-specfun,
octave-vrml,
opencolorio,
openvdb,
pescetti,
php-guzzlehttp,
proofgeneral,
pyblosxom,
pyopencl,
pyqi,
python-expyriment,
python-flask-httpauth,
python-mzml,
python-simpy,
python-tidylib,
reactive-streams,
scmxx,
shared-mime-info,
sikuli,
siproxd,
srtp,
tachyon,
tcltk-defaults,
urjtag,
velvet.
The following packages became reproducible after getting fixed:
C
when sorting source file list.debian/changelog
entry in build string..pyc
files..pyc
files.debian/changelog
entry.debian/changelog
entry as build time.debian/changelog
entry as build time.Build.PL
.debian/changelog
entry as build time.--fuzzy-threshold
option to specify the TLSH score used as cut-off
for fuzzy matching. Specifying 0
will disable fuzzy-matching entirely.
Suggested by Jakub Wilk.--new-file
option to treat absent files as empty. This make diffoscope a great
tool to look at the content of an archive at once by comparing it with a non-existent
file (example).
Suggested by Jakub Wilk.--help
..file
assembler directive can help with random filenames in debug symbols.
Package reviews
235 reviews have
been removed, 84 added and 277 updated this week.
29 new FTBFS bugs were filled by Chris Lamb, Chris West (Faux), Daniel Stender, and Niko Tyni.
New issues identified this week: random_order_in_ibus_table_createdb_output, random_order_in_antlr_output, nondetermistic_link_order_in_module_build, and timestamps_in_tex_documents.
Misc.
Thanks to Dhole and Thomas Vincent, the talk held at DebConf15 now has subtitles!
Void Linux started to merge changes to make packages produced by xbps reproducible.
Provides
field.
Lunar rebased the pu/reproducible_builds
branch for dpkg on top of the released 1.18.2. This made visible an issue with udeb
s and automatically generated debug packages.
The summary from the meeting at DebConf15 between ftpmasters, dpkg mainatainers and reproducible builds folks has been posted to the revelant mailing lists.
Packages fixed
The following 70 packages became reproducible due to changes in their
build dependencies:
activemq-activeio,
async-http-client,
classworlds,
clirr,
compress-lzf,
dbus-c++,
felix-bundlerepository,
felix-framework,
felix-gogo-command,
felix-gogo-runtime,
felix-gogo-shell,
felix-main,
felix-shell-tui,
felix-shell,
findbugs-bcel,
gco,
gdebi,
gecode,
geronimo-ejb-3.2-spec,
git-repair,
gmetric4j,
gs-collections,
hawtbuf,
hawtdispatch,
jack-tools,
jackson-dataformat-cbor,
jackson-dataformat-yaml,
jackson-module-jaxb-annotations,
jmxetric,
json-simple,
kryo-serializers,
lhapdf,
libccrtp,
libclaw,
libcommoncpp2,
libftdi1,
libjboss-marshalling-java,
libmimic,
libphysfs,
libxstream-java,
limereg,
maven-debian-helper,
maven-filtering,
maven-invoker,
mochiweb,
mongo-java-driver,
mqtt-client,
netty-3.9,
openhft-chronicle-queue,
openhft-compiler,
openhft-lang,
pavucontrol,
plexus-ant-factory,
plexus-archiver,
plexus-bsh-factory,
plexus-cdc,
plexus-classworlds2,
plexus-component-metadata,
plexus-container-default,
plexus-io,
pytone,
scolasync,
sisu-ioc,
snappy-java,
spatial4j-0.4,
tika,
treeline,
wss4j,
xtalk,
zshdb.
The following packages became reproducible after getting fixed:
pybuild
to get rid of .pyc
files.SOURCE_DATE_EPOCH
.dpkg-parsechangelog
in debian/rules
.SOURCE_DATE_EPOCH
for version string.SOURCE_DATE_EPOCH
to set manpage date.scm-safe
which tells ResourceGen
that no timestamps should be included.SOURCE_DATE_EPOCH
have been improved to support systems without GNU date
.
reproducible.debian.net
armhf
is finally being tested, which also means the remote building of Debian packages finally works! This paves the way to perform the tests on even more architectures and doing variations on CPU and date. Some packages even produce the same binary Arch:all
packages on different architectures (1, 2). (h01ger)
Tests for FreeBSD are finally running. (h01ger)
As it seems the gcc5 transition has cooled off, we schedule sid more often than testing again on amd64
. (h01ger)
disorderfs has been built and installed on all build nodes (amd64
and armhf
). One issue related to permissions for root and unpriviliged users needs to be solved before disorderfs
can be used on reproducible.debian.net. (h01ger)
strip-nondeterminism
Version 0.011-1 has been released on August 29th. The new version updates dh_strip_nondeterminism
to match recent changes in debhelper
. (Andrew Ayer)
disorderfs
disorderfs, the new FUSE filesystem to ease testing of filesystem-related variations, is now almost ready to be used. Version 0.2.0 adds support for extended attributes. Since then Andrew Ayer also added support to reverse directory entries instead of shuffling them, and arbitrary padding to the number of blocks used by files.
Package reviews
142 reviews have
been removed, 48 added and 259 updated this week.
Santiago Vila renamed the not_using_dh_builddeb
issue into varying_mtimes_in_data_tar_gz_or_control_tar_gz to align better with other tag names.
New issue identified this week: random_order_in_python_doit_completion.
37 FTBFS issues have been reported by Chris West (Faux) and Chris Lamb.
Misc.
h01ger gave a talk at FrOSCon on August 23rd. Recordings are already online.
These reports are being reviewed and enhanced every week by many people hanging out on #debian-reproducible
. Huge thanks!
__repr__
so memory addresses don't appear in docs (#795826). Patches by Val Lorentz.erlc
. Patch by Chris West (Faux) and Chris Lamb..file
to the assembler output.-d
option to txt2man
and add the --date
option to override the current date.SOURCE_DATE_EPOCH
instead of the custom WHEEL_FORCE_TIMESTAMP
. akira sent one making man2html SOURCE_DATE_EPOCH
aware.
St phane Glondu reported that dpkg-source would not respect tarball permissions when unpacking under a umask
of 002
.
After hours of iterative testing during the DebConf workshop, Sandro Knau created a test case showing how pdflatex
output can be non-deterministic with some PNG files.
Packages fixed
The following 65 packages became reproducible due to changes in their
build dependencies:
alacarte,
arbtt,
bullet,
ccfits,
commons-daemon,
crack-attack,
d-conf,
ejabberd-contrib,
erlang-bear,
erlang-cherly,
erlang-cowlib,
erlang-folsom,
erlang-goldrush,
erlang-ibrowse,
erlang-jiffy,
erlang-lager,
erlang-lhttpc,
erlang-meck,
erlang-p1-cache-tab,
erlang-p1-iconv,
erlang-p1-logger,
erlang-p1-mysql,
erlang-p1-pam,
erlang-p1-pgsql,
erlang-p1-sip,
erlang-p1-stringprep,
erlang-p1-stun,
erlang-p1-tls,
erlang-p1-utils,
erlang-p1-xml,
erlang-p1-yaml,
erlang-p1-zlib,
erlang-ranch,
erlang-redis-client,
erlang-uuid,
freecontact,
givaro,
glade,
gnome-shell,
gupnp,
gvfs,
htseq,
jags,
jana,
knot,
libconfig,
libkolab,
libmatio,
libvsqlitepp,
mpmath,
octave-zenity,
openigtlink,
paman,
pisa,
pynifti,
qof,
ruby-blankslate,
ruby-xml-simple,
timingframework,
trace-cmd,
tsung,
wings3d,
xdg-user-dirs,
xz-utils,
zpspell.
The following packages became reproducible after getting fixed:
debian/changelog
entry.debian/changelog
entry.LC_ALL
set to C
.debian/changelog
entry.LC_ALL
set to C
.lib/Lucy.xs
in a deterministic order.LC_ALL
set to C
.aff
files generated by mk_he_affix
.icalderivedvalue.c
.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.U
flag to ar
.
Reiner Herrmann reported an issue with pound which embeds random dhparams
in its code during the build. Better solutions are yet to be found.
reproducible.debian.net
Package pages on reproducible.debian.net now have a new layout improving readability designed by Mattia Rizzolo, h01ger, and Ulrike. The navigation is now on the left as vertical space is more valuable nowadays.
armhf
is now enabled on all pages except the dashboard. Actual tests on armhf
are expected to start shortly. (Mattia Rizzolo, h01ger)
The limit on how many packages people can schedule using the reschedule
script on Alioth has been bumped to 200. (h01ger)
mod_rewrite
is now used instead of JavaScript for the form in the dashboard. (h01ger)
Following the rename of the software, debbindiff has mostly been replaced by either diffoscope or differences in generated HTML and IRC notification output.
Connections to UDD have been made more robust. (Mattia Rizzolo)
diffoscope development
diffoscope version 31 was released on August 21st. This version improves fuzzy-matching by using the tlsh algorithm instead of ssdeep.
New command line options are available: --max-diff-input-lines
and --max-diff-block-lines
to override limits on diff
input and output (Reiner Herrmann), --debugger
to dump the user into pdb in case of crashes (Mattia Rizzolo).
jar
archives should now be detected properly (Reiner Herrman). Several general code cleanups were also done by Chris Lamb.
strip-nondeterminism development
Andrew Ayer released strip-nondeterminism version 0.010-1. Java properties file in jar should now be detected more accurately. A missing dependency spotted by St phane Glondu has been added.
Testing directory ordering issues: disorderfs
During the reproducible builds workshop at DebConf, participants identified that we were still short of a good way to test variations on filesystem behaviors (e.g. file ordering or disk usage). Andrew Ayer took a couple of hours to create disorderfs. Based on FUSE, disorderfs in an overlay filesystem that will mount the content of a directory at another location. For this first version, it will make the order in which files appear in a directory random.
Documentation update
Dhole documented how to implement support for SOURCE_DATE_EPOCH
in Python, bash, Makefiles, CMake, and C.
Chris Lamb started to convert the wiki page describing SOURCE_DATE_EPOCH
into a Freedesktop-like specification in the hope that it will convince more upstream to adopt it.
Package reviews
44 reviews have
been removed, 192 added and 77 updated this week.
New issues identified this week: locale_dependent_order_in_devlibs_depends, randomness_in_ocaml_startup_files, randomness_in_ocaml_packed_libraries, randomness_in_ocaml_custom_executables, undeterministic_symlinking_by_rdfind, random_build_path_by_golang_compiler, and images_in_pdf_generated_by_latex.
117 new FTBFS bugs have been reported by Chris Lamb, Chris West (Faux), and Niko Tyni.
Misc.
Some reproducibility issues might face us very late. Chris Lamb noticed that the test suite for python-pykmip was now failing because its test certificates have expired. Let's hope no packages are hiding a certificate valid for 10 years somewhere in their source!
Pictures courtesy and copyright of Debian's own paparazzi: Aigars Mahinovs.
ocamldoc
to build reproducible manpages using a patch by Valentin Lorentz.DEBIANDOC_DATE
environment variable to override the content of the <date>
tag.PODDATE
to the date of the latest debian/changelog
entry.pod2man
to use the date of the latest debian/changelog
entry.SOURCE_DATE_EPOCH
as source for the manpage date instead of the currentdate.TZ
to UTC
when using zip
.grep
to cope with non-UTF8 files.SOURCE_DATE_EPOCH
as source for the manpage date instead of the currentdate.TZ=UTC
in debian/rules
.debian/control
file with all locales. Original patch by Chris Lamb.SOURCE_DATE_EPOCH
. She uploded a package with the enhancement to the experimental reproducible repository.
Packages fixed
The following 15 packages became reproducible due to changes in their
build dependencies:
dracut,
editorconfig-core,
elasticsearch,
fish,
libftdi1,
liblouisxml,
mk-configure,
nanoc,
octave-bim,
octave-data-smoothing,
octave-financial,
octave-ga,
octave-missing-functions,
octave-secs1d,
octave-splines,
valgrind.
The following packages became reproducible after getting fixed:
debian/changelog
entry.debian/changelog
entry in manpage.SOURCE_DATE_EPOCH
.SOURCE_DATE_EPOCH
.debian/changelog
entry.armhf
build hosts were provided by Vagrant Cascadian and have been configured to be used by jenkins.debian.net. Work on including armhf
builds in the reproducible.debian.net webpages has begun. So far the repository comparison page just shows us which armhf
binary packages are currently missing in our repo. (h01ger)
The scheduler has been changed to re-schedule more packages from stretch than sid, as the gcc5 transition has started This mostly affects build log age. (h01ger)
A new depwait status has been introduced for packages which can't be built because of missing build dependencies. (Mattia Rizzolo)
debbindiff development
Finally, on August 31st, Lunar released debbindiff 27 containing a complete overhaul of the code for the comparison stage. The new architecture is more versatile and extensible while minimizing code duplication. libarchive is now used to handle cpio archives and iso9660 images through the newly packaged python-libarchive-c. This should also help support a couple other archive formats in the future. Symlinks and devices are now properly compared. Text files are compared as Unicode after being decoded, and encoding differences are reported. Support for Sqlite3 and Mono/.NET executables has been added. Thanks to Valentin Lorentz, the test suite should now run on more systems. A small defiency in unquashfs has been identified in the process. A long standing optimization is now performed on Debian package: based on the content of the md5sums
control file, we skip comparing files with matching hashes. This makes debbindiff usable on packages with many files. Fuzzy-matching is now performed for files in the same container (like a tarball) to handle renames. Also, for Debian .changes
, listed files are now compared without looking the embedded version number. This makes debbindiff a lot more useful when comparing different versions of the same package.
Based on the rearchitecturing work has been done to allow parallel processing. The branch now seems to work most of the time. More test needs to be done before it can be merged.
The current fuzzy-matching algorithm, ssdeep, has showed disappointing results. One important use case is being able to properly compare debug symbols. Their path is made using the Build ID. As this identifier is made with a checksum of the binary content, finding things like CPP macros is much easier when a diff of the debug symbols is available. Good news is that TLSH, another fuzzy-matching algorithm, has been tested with much better results. A package is waiting in NEW and the code is ready for it to become available.
A follow-up release 28 was made on August 2nd fixing content label used for gzip2, bzip2 and xz files and an error on text files only differing in their encoding. It also contains a small code improvement on how comments on Difference
object are handled.
This is the last release name debbindiff
. A new name has been chosen to better reflect that it is not a Debian specific tool. Stay tuned!
Documentation update
Valentin Lorentz updated the patch submission template to suggest to write the kind of issue in the bug subject.
Small progress have been made on the Reproducible Builds HOWTO while preparing the related CCCamp15 talk.
Package reviews
235 obsolete
reviews have
been removed, 47 added and 113 updated this week.
42 reports for packages failing to build from source have been made by Chris West (Faux).
New issue added this week: haskell_devscripts_locale_substvars.
Misc.
Valentin Lorentz wrote a script to report packages tested as unreproducible installed on a system. We encourage everyone to run it on their systems and give feedback!
DEB_CHANGELOG_DATETIME
with non English locales.SOURCE_DATE_EPOCH
and use UTC as a timezone. A modified package is now being experimented.
Packages fixed
The following 14 packages became reproducible due to changes in their
build dependencies:
bino,
cfengine2,
fwknop,
gnome-software,
jnr-constants,
libextractor,
libgtop2,
maven-compiler-plugin,
mk-configure,
nanoc,
octave-splines,
octave-symbolic,
riece,
vdr-plugin-infosatepg.
The following packages became reproducible after getting fixed:
debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.debian/changelog
entry.TZ=UTC
when calling zip
.TZ=UTC
when calling zip
.TZ=UTC
when calling zip
.mdate-sh
deterministic. Original patch by Reiner Herrmann.
Kenneth J. Pronovici uploaded epydoc/3.0.1+dfsg-8 which now honors SOURCE_DATE_EPOCH. Original patch by Reiner Herrmann.
Chris Lamb submitted a patch to dh-python to make the order of the generated maintainer scripts deterministic. Chris also offered a fix for a source of non-determinism in dpkg-shlibdeps when packages have alternative dependencies.
Dhole provided a patch to add support for SOURCE_DATE_EPOCH
to gettext.
Packages fixed
The following 78 packages became reproducible in our setup due to changes in their
build dependencies:
chemical-mime-data,
clojure-contrib,
cobertura-maven-plugin,
cpm,
davical,
debian-security-support,
dfc,
diction,
dvdwizard,
galternatives,
gentlyweb-utils,
gifticlib,
gmtkbabel,
gnuplot-mode,
gplanarity,
gpodder,
gtg-trace,
gyoto,
highlight.js,
htp,
ibus-table,
impressive,
jags,
jansi-native,
jnr-constants,
jthread,
jwm,
khronos-api,
latex-coffee-stains,
latex-make,
latex2rtf,
latexdiff,
libcrcutil,
libdc0,
libdc1394-22,
libidn2-0,
libint,
libjava-jdbc-clojure,
libkryo-java,
libphone-ui-shr,
libpicocontainer-java,
libraw1394,
librostlab-blast,
librostlab,
libshevek,
libstxxl,
libtools-logging-clojure,
libtools-macro-clojure,
litl,
londonlaw,
ltsp,
macsyfinder,
mapnik,
maven-compiler-plugin,
mc,
microdc2,
miniupnpd,
monajat,
navit,
pdmenu,
pirl,
plm,
scikit-learn,
snp-sites,
sra-sdk,
sunpinyin,
tilda,
vdr-plugin-dvd,
vdr-plugin-epgsearch,
vdr-plugin-remote,
vdr-plugin-spider,
vdr-plugin-streamdev,
vdr-plugin-sudoku,
vdr-plugin-xineliboutput,
veromix,
voxbo,
xaos,
xbae.
The following packages became reproducible after getting fixed:
LC_ALL=C
when running sort
.TZ=UTC
when calling unzip
.Makefile
.debian/changelog
in version string.TZ=UTC
when calling unzip
.TZ=UTC
when calling unzip
.TZ=UTC
when calling unzip
.TZ=UTC
when calling unzip
.TZ=UTC
when calling unzip
.debian/changelog
in manpages.*.pyo
and *.pyc
from binary package.debian/changelog
when generating version strings.debian/changelog
.freebsd-hackers
mailing list. The build is run on a new virtual machine running FreeBSD 10.1 with 3 cores and 6 GB of RAM, also sponsored by Profitbricks.
strip-nondeterminism development
Andrew Ayer released version 0.009 of strip-nondeterminism. The new version will strip locales from Javadoc, include the name of files causing errors, and ignore unhandled (but rare) zip64 archives.
debbindiff development
Lunar continued its major refactoring to enhance code reuse and pave the way to fuzzy-matching and parallel processing. Most file comparators have now been converted to the new class hierarchy.
In order to support for archive formats, work has started on packaging Python bindings for libarchive. While getting support for more archive formats with a common interface is very nice, libarchive is a stream oriented library and might have bad performance with how debbindiff currently works. Time will tell if better solutions need to be found.
Documentation update
Lunar started a Reproducible builds HOWTO intended to explain the different aspects of making software build reproducibly to the different audiences that might have to get involved like software authors, producers of binary packages, and distributors.
Package reviews
17 obsolete
reviews have
been removed, 212 added and 46 updated this week.
15 new bugs for packages failing to build from sources have been reported by Chris West (Faux), and Mattia Rizzolo.
Presentations
Lunar presented Debian efforts and some recipes on making software build reproducibly at Libre Software Meeting 2015. Slides and a video recording are available.
Misc.
h01ger, dkg, and Lunar attended a Core Infrastructure Initiative meeting. The progress and tools mode for the Debian efforts were shown. Several discussions also helped getting a better understanding of the needs of other free software projects regarding reproducible builds. The idea of a global append only log, similar to the logs used for Certificate Transparency, came up on multiple occasions. Using such append only logs for keeping records of sources and build results has gotten the name Binary Transparency Logs . They would at least help identifying a compromised software signing key. Whether the benefits in using such logs justify the costs need more research.
Debian is undertaking a huge effort to develop a reproducible builds system. I'd like to thank you for that. This could be Debian's most important project, with how badly computer security has been going.
PerniciousPunk in Reddit's Ask me anything! to Neil McGovern, DPL. What happened in the reproducible builds effort this week: Toolchain fixes More tools are getting patched to use the value of the SOURCE_DATE_EPOCH environment variable as the current time:
SOURCE_DATE_EPOCH
to the time of the latest debian/changelog
entry when exporting build flags, patch sent as #791823 (Dhole),texlive-bin
(akira) and libxslt
(Dhole) with the aforementioned support for SOURCE_DATE_EPOCH
.debhelper
exported TZ=UTC
and this made packages capturing the current date (without the time) reproducible in the current test environment.
The following packages became reproducible after getting fixed:
debian/changelog
date in the manpage.debian/changelog
date as build date and use debian
as the builder hostname.debian/changelog
date as bui
ld date.reproducible.debian.net
.
Next.