In considering storage trends for the consumer side I m looking at the current prices from MSY (where I usually buy computer parts). I know that other stores will have slightly different prices but they should be very similar as they all have low margins and wholesale prices are the main factor. Small Hard Drives Aren t Viable The cheapest hard drive that MSY sells is $68 for 500G of storage. The cheapest SSD is $49 for 120G and the second cheapest is $59 for 240G. SSD is cheaper at the low end and significantly faster. If someone needed about 500G of storage there s a 480G SSD for $97 which costs $29 more than a hard drive. With a modern PC if you have no hard drives you will notice that it s quieter. For anyone who s buying a new PC spending an extra $29 is definitely worthwhile for the performance, low power use, and silence. The cheapest 1TB disk is $69 and the cheapest 1TB SSD is $159. Saving $90 on the cost of a new PC probably isn t worth while. For 2TB of storage the cheapest options are Samsung NVMe for $339, Crucial SSD for $335, or a hard drive for $95. Some people would choose to save $244 by getting a hard drive instead of NVMe, but if you are getting a whole system then allocating $244 to NVMe instead of a faster CPU would probably give more benefits overall. Computer stores typically have small margins and computer parts tend to quickly either become cheaper or be obsoleted by better parts. So stores don t want to stock parts unless they will sell quickly. Disks smaller than 2TB probably aren t going to be profitable for stores for very long. The trend of SSD and NVMe becoming cheaper is going to make 2TB disks non-viable in the near future. NVMe vs SSD M.2 NVMe devices are at comparable prices to SATA SSDs. For some combinations of quality and capacity NVMe is about 50% more expensive and for some it s slightly cheaper (EG Intel 1TB NVMe being cheaper than Samsung EVO 1TB SSD). Last time I checked about half the motherboards on sale had a single M.2 socket so for a new workstation that doesn t need more than 2TB of storage (the largest NVMe that MSY sells) it wouldn t make sense to use anything other than NVMe. The benefit of NVMe is NOT throughput (even though NVMe devices can often sustain over 4GB/s), it s low latency. Workstations can t properly take advantage of this because RAM is so cheap ($198 for 32G of DDR4) that compiles etc mostly come from cache and because most filesystem writes on workstations aren t synchronous. For servers a large portion of writes are synchronous, for example a mail server can t acknowledge receiving mail until it knows that it s really on disk, so there s a lot of small writes that block server processes and the low latency of NVMe really improves performance. If you are doing a big compile on a workstation (the most common workstation task that uses a lot of disk IO) then the writes aren t synchronised to disk and if the system crashes you will just do all the compilation again. While NVMe doesn t give a lot of benefit over SSD for workstation use (I ve uses laptops with SSD and NVMe and not noticed a great difference) of course I still want better performance. ;) Last time I checked I couldn t easily buy a PCIe card that supported 2*NVMe cards, I m sure they are available somewhere but it would take longer to get and probably cost significantly more than twice as much. That means a RAID-1 of NVMe takes 2 PCIe slots if you don t have an M.2 socket on the motherboard. This was OK when I installed 2*NVMe devices on a server that had 18 disks and lots of spare PCIe slots. But for some systems PCIe slots are an issue. My home server has all PCIe slots used by a video card and Ethernet cards and the BIOS probably won t support booting from NVMe. It s a Dell server so I can t just replace the motherboard with one that has more PCIe slots and M.2 on the motherboard. As it s running nicely and doesn t need replacing any time soon I won t be using NVMe for home server stuff. Small Servers Most servers that I am responsible for have less than 2TB of storage. For my clients I now only recommend SSD storage for small servers and am recommending SSD for replacing any failed disks. My home server has 2*500G SSDs in a BTRFS RAID-1 for the root filesystem, and 3*4TB disks in a BTRFS RAID-1 for storing big files. I bought the SSDs when 500G SSDs were about $250 each and bought 2*4TB disks when they were about $350 each. Currently that server has about 3.3TB of space used and I could probably get it down to about 2.5TB if I deleted things I don t really need. If I was getting storage for that server now I d use 2*2TB SSDs and 3*1TB hard drives for the stuff that doesn t fit on SSDs (I have some spare 1TB disks that came with servers). If I didn t have spare hard drives I d get 3*2TB SSDs for that sort of server which would give 3TB of BTRFS RAID-1 storage. Last time I checked Dell servers had a card for supporting M.2 as an optional extra so Dells probably won t boot from NVMe without extra expense. Ars Technica has an informative article about WD selling SMR disks as NAS disks [1]. The Shingled Magnetic Recording technology allows greater storage density on a platter which leads to either larger capacity or cheaper disks but at the cost of lower write performance and apparently extremely bad latency in some situations. NAS disks are supposed to be low latency as the expectation is that they will be used in a RAID array and kicked out of the array if they have problems. There are reports of ZFS kicking SMR disks from RAID sets. I think this will end the use of hard drives for small servers. For a server you don t want to deal with this sort of thing, by definition when a server goes down multiple people will stop work (small server implies no clustering). Spending extra to get SSDs just to avoid the risk of unexpected SMR would be a good plan. Medium Servers The largest SSD and NVMe devices that are readily available are 2TB but 10TB disks are commodity items, there are reports of 20TB hard drives being available but I can t find anyone in Australia selling them. If you need to store dozens or hundreds of terabytes than hard drives have to be part of the mix at this time. There s no technical reason why SSDs larger than 10TB can t be made (the 2.5 SATA form factor has more than 5* the volume of a 2TB M.2 card) and it s likely that someone sells them outside the channels I buy from, but probably at a price higher than what my clients are willing to pay. If you want 100TB of affordable storage then a mid range server like the Dell PowerEdge T640 which can have up to 18*3.5 disks is good. One of my clients has a PowerEdge T630 with 18*3.5 disks in the 8TB-10TB range (we replace failed disks with the largest new commodity disks available, it used to have 6TB disks). ZFS version 0.8 introduced a Special VDEV Class which stores metadata and possibly small data blocks on faster media. So you could have some RAID-Z groups on hard drives for large storage and the metadata on a RAID-1 on NVMe for fast performance. For medium size arrays on hard drives having a find / operation take hours is not uncommon, for large arrays having it take days isn t that uncommon. So far it seems that ZFS is the only filesystem to have taken the obvious step of storing metadata on SSD/NVMe while bulk data is on cheap large disks. One problem with large arrays is that the vibration of disks can affect the performance and reliability of nearby disks. The ZFS server I run with 18 disks was originally setup with disks from smaller servers that never had ZFS checksum errors, but when disks from 2 small servers were put in one medium size server they started getting checksum errors presumably due to vibration. This alone is a sufficient reason for paying a premium for SSD storage. Currently the cost of 2TB of SSD or NVMe is between the prices of 6TB and 8TB hard drives, and the ratio of price/capacity for SSD and NVMe is improving dramatically while the increase in hard drive capacity is slow. 4TB SSDs are available for $895 compared to a 10TB hard drive for $549, so it s 4* more expensive on a price per TB. This is probably good for Windows systems, but for Linux systems where ZFS and special VDEVs is an option it s probably not worth considering. Most Linux user cases where 4TB SSDs would work well would be better served by smaller NVMe and 10TB disks running ZFS. I don t think that 4TB SSDs are at all popular at the moment (MSY doesn t stock them), but prices will come down and they will become common soon enough. Probably by the end of the year SSDs will halve in price and no hard drives less than 4TB will be viable. For rack mounted servers 2.5 disks have been popular for a long time. It s common for vendors to offer 2 versions of a rack mount server for 2.5 and 3.5 disks where the 2.5 version takes twice as many disks. If the issue is total storage in a server 4TB SSDs can give the same capacity as 8TB HDDs. SMR vs Regular Hard Drives Rumour has it that you can buy 20TB SMR disks, I haven t been able to find a reference to anyone who s selling them in Australia (please comment if you know who sells them and especially if you know the price). I expect that the ZFS developers will soon develop a work-around to solve the problems with SMR disks. Then arrays of 20TB SMR disks with NVMe for special VDEVs will be an interesting possibility for storage. I expect that SMR disks will be the majority of the hard drive market by 2023 if hard drives are still on the market. SSDs will be large enough and cheap enough that only SMR disks will offer enough capacity to be worth using. I think that it is a possibility that hard drives won t be manufactured in a few years. The volume of a 3.5 disk is significantly greater than that of 10 M.2 devices so current technology obviously allows 20TB of NVMe or SSD storage in the space of a 3.5 disk. If the price of 16TB NVMe and SSD devices comes down enough (to perhaps 3* the price of a 20TB hard drive) almost no-one would want the hard drive and it wouldn t be viable to manufacture them. It s not impossible that in a few years time 3D XPoint and similar fast NVM technologies occupy the first level of storage (the ZFS special VDEV , OS swap device, log device for database servers, etc) and NVMe occupies the level for bulk storage with no space left in the market for spinning media. Computer Cases For servers I expect that models supporting 3.5 storage devices will disappear. A 1RU server with 8*2.5 storage devices or a 2RU server with 16*2.5 storage devices will probably be of use to more people than a 1RU server with 4*3.5 or a 2RU server with 8*3.5 . My first IBM PC compatible system had a 5.25 hard drive, a 5.25 floppy drive, and a 3.5 floppy drive in 1988. My current PC is almost a similar size and has a DVD drive (that I almost never use) 5 other 5.25 drive bays that have never been used, and 5*3.5 drive bays that I have never used (I have only used 2.5 SSDs). It would make more sense to have PC cases designed around 2.5 and maybe 3.5 drives with no more than one 5.25 drive bay. The Intel NUC SFF PCs are going in the right direction. Many of them only have a single storage device but some of them have 2*M.2 sockets allowing RAID-1 of NVMe and some of them support ECC RAM so they could be used as small servers. A USB DVD drive costs $36, it doesn t make sense to have every PC designed around the size of an internal DVD drive that will probably only be used to install the OS when a $36 USB DVD drive can be used for every PC you own. The only reason I don t have a NUC for my personal workstation is that I get my workstations from e-waste. If I was going to pay for a PC then a NUC is the sort of thing I d pay to have on my desk.

• [1] http://tinyurl.com/y8gglf5c

LWN published today yet another great piece of writing, Loaded terms in free software. I am sorry, the content will not be immediately available to anybody following at home, as LWN is based on a subscription model But a week from now, the article will be open for anybody to read. Or you can ask me (you most likely can find my contact addresses, as they are basically everywhere) for a subscriber link, I will happily provide it. In consonance with the current mood that started with the killing of George Floyd and sparked worldwide revolts against police brutality, racism (mostly related to police and law enforcement forces, but social as well) and the like, the debate that already started some months ago in technical communities has re-sparked: We have many terms that come with long histories attached to them, and we are usually oblivious to their obvious meaning. We? Yes, we, the main users and creators of technology. I never felt using master and slave to refer to different points of a protocol, bus, clock or whatever (do refer to the Wikipedia article for a fuller explanation) had any negative connotations but then again, those terms have never tainted my personal family. That is, I understand I speak from a position of privilege. A similar although less heated issue goes around the blacklist and whitelist terms, or other uses that use white to refer to good, law-abiding citizens, and black to refer to somewhat antisocial uses (i.e. the white hat and black hat hackers). For several years, this debate has been sparking and dying off. Some important changes have been made Particularly, in 2017 the Internet Software Consortium started recommending Primary and Secondary, Python dropped master/slave pairs after a quite thorough and deep review throughout 2018, GitHub changed the default branch from master to main earlier this week. The Internet Engineering Task Force has a draft (that lapsed and thus sadly didn t become an RFC, but still, is archived), Terminology, Power and Oppressive Language that lists suggested alternatives:

There are also many other relationships that can be used as metaphors, Eglash s research calls into question the accuracy of the master-slave metaphor. Fortunately, there are ample alternatives for the master-slave relationship. Several options are suggested here and should be chosen based on the pairing that is most clear in context:

• Primary-secondary
• Leader-follower
• Active-standby
• Primary-replica
• Writer-reader
• Coordinator-worker
• Parent-helper

I ll add that I think we Spanish-speakers are not fully aware of the issue s importance, because the most common translation I have seen for master/slave is maestro/esclavo: Maestro is the word for teacher (although we do keep our slaves in place). But think whether it sounds any worse if you refer to device pairs, or members of a database high-availability cluster, or whatever as Amo and Esclavo. It does sound much worse I cannot add much of value to this debate. I am just happy issues like this are being recognized and dealt with. If the topic interests you, do refer to the LWN article! Some excrepts: I consider the following to be the core of Jonathan Corbet s writeup:

Recent events, though, have made it clear even to those of us who were happy to not question this view that the story of slavery and the wider racist systems around it is not yet finished. There are many people who are still living in the middle of it, and it is not a nice place to be. We are not so enlightened as we like to think we are. If there is no other lesson from the events of the last few weeks, we should certainly take to heart the point that we need to be listening to the people who have been saying, for many years, that they are still suffering. If there are people who are telling us that terms like slave or blacklist are a hurtful reminder of the inequities that persist in our society, we need to accept that as the truth and act upon it. Etymological discussions on what, say, master really means may be interesting, but they miss the point and are irrelevant to this discussion.

Part of a comment by user yokem_55:

Often, it seems to me that the replacement words are much more descriptive and precise than the old language. Allowlist is far more obviously a list of explicitly authorized entities than whitelist . Mainline has a more obvious meaning of a core stream of development than master . The benefit of moving past this language is more than just changing cultural norms, it s better, more precise communication across the board.

Another spot-on comment, by user alan:

From my perspective as a Black American male, I think that it s nice to see people willing to see and address racism in various spheres. I am concerned that some of these steps will be more performative than substantial. Terminology changes in software so as to be more welcoming is a nice thing. Ensuring that oppressed minorities have access to the tools and resources to help reduce inequity and ensuring equal protection under the laws is better. We ll get there one day I m sure. The current ask is much simpler, its just to stop randomly killing and terrorizing us. Please and thank you.

So Maybe the protests of this year caught special notoriety because the society is reacting after (or during, for many of us) the lockdown. In any case, I hope for their success in changing the planet s culture of oppression.

Comments Tomas Janousek 2020-06-19 10:04:32 +0200 In the blog post On masters and slaves, whitelists and blacklists you claim that GitHub changed the default branch from master to main earlier this week but I don t think that change is in effect yet. When you create a repo, the default branch is still named master . Gunnar Wolf 2020-06-19 11:52:30 -0500 Umh, seems you are right. Well, what can I say? I m reporting only what I have been able to find / read Now, given that said master branch does not carry any Git-specific meaning and is just a commonly used configuration I hope people start picking it up. No, I have not renamed master branches in any of my repos but intend to do so soonish. Tomas Janousek 2020-06-19 20:01:52 +0200 Yeah, don t worry. I just find it sad that so much inaccurate news is spreading from a single CEO tweet, and I wanted to help stop that. I m sure some change will happen eventually, but until it does, we shouldn t speak about it in the past tense. :-)

This post was originally published in the Wikimedia Tech blog, and is authored by Arturo Borrero Gonzalez and Brooke Storm. In the previous post, we shared the context on the recent Kubernetes upgrade that we introduced in the Toolforge service. Today we would like to dive a bit more in the technical details. Custom admission controllers One of the key components of the Toolforge Kubernetes are our custom admission controllers. We use them to validate and enforce that the usage of the service is what we intended for. Basically, we have 2 of them:

• Ingress admission controller [source code]
• Registry admission controller [source code]

The source code is written in Golang, which is pretty convenient for natively working in a Kubernetes environment. Both code repositories include extensive documentation: how to develop, test, use, and deploy them. We decided to go with custom admission controllers because we couldn t find any native (or built-in) Kubernetes mechanism to accomplish the same sort of checks on user activity. With the Ingress controller, we want to ensure that Ingress objects only handle traffic to our internal domains, which by the time of this writing, are toolforge.org (our new domain) and tools.wmflabs.org (legacy). We safe-list the kube-system namespace and the tool-fourohfour namespace because both need special consideration. More on the Ingress setup later. The registry controller is pretty simple as well. It ensures that only our internal docker registry is used for user-scheduled containers running in Kubernetes. Again, we exclude from the checks containers running in the kube-system namespace (those used by Kubernetes itself). Other than that, the validation itself is pretty easy. For some extra containers we run (like those related to Prometheus metrics) what we do is simply upload those docker images to our internal registry. The controls provided by this admission controller helps us validate that only FLOSS software is run in our environment, which is one of the core rules of Toolforge. RBAC and Pod Security Policy setup I would like to comment next on our RBAC and Pod Security Policy setup. Using the Pod Security Policies (or PSP) we establish a set of constraints on what containers can and can t do in our cluster. We have many PSP configured in our setup:

• Privileged policy: used by Kubernetes containers themselves basically a very relaxed set of constraints that are required for the system itself to work.
• Default policy: a bit more restricted than the privileged policy, is intended for admins to deploy services, but it isn t currently in use..
• Toolforge user policies: this applies to user-scheduled containers, and there are some obvious restrictions here: we only allow unprivileged pods, we control which HostPath is available for pods, use only default Linux capabilities, etc.

Each user can interact with their own namespace (this is how we achieve multi-tenancy in the cluster). Kubernetes knows about each user by means of TLS certs, and for that we have RBAC. Each user has a rolebinding to a shared cluster-role that defines how Toolforge tools can use the Kubernetes API. The following diagram shows the design of our RBAC and PSP in our cluster: RBAC and PSP for Toolforge, original image in wikitech I mentioned that we know about each user by means of TLS certificates. This is true, and in fact, there is a key component in our setup called maintain-kubeusers. This custom piece of Python software is run as a pod inside the cluster and is responsible for reading our external user database (LDAP) and generating the required credentials, namespaces, and other configuration bits for them. With the TLS cert, we basically create a kubeconfig file that is then written into the homes NFS share, so each Toolforge user has it in their shell home directory. Networking and Ingress setup With the basic security controls in place, we can move on to explaining our networking and Ingress setup. Yes, the Ingress word might be a bit overloaded already, but we refer here to Ingress as the path that end-users follow from their web browser in their local machine to a webservice running in the Toolforge cluster. Some additional context here. Toolforge is not only Kubernetes, but we also have a Son of GridEngine deployment, a job scheduler that covers some features not available in Kubernetes. The grid can also run webservices, although we are encouraging users to migrate them to Kubernetes. For compatibility reasons, we needed to adapt our Ingress setup to accommodate the old web grid. Deciding the layout of the network and Ingress was definitely something that took us some time to figure out because there is not a single way to do it right. The following diagram can be used to explain the different steps involved in serving a web service running in the new Toolforge Kubernetes. Toolforge k8s network topology, original image in Wikitech The end-user HTTP/HTTPs request first hits our front proxy in (1). Running here is NGINX with a custom piece of LUA code that is able to decide whether to contact the web grid or the new Kubernetes cluster. TLS termination happens here as well, for both domains (toolforge.org and tools.wmflabs.org). Note this proxy is reachable from the internet, as it uses a public IPv4 address, a floating IP from CloudVPS, the infrastructure service we provide based on Openstack. Remember that our Kubernetes is directly built in virtual machines a bare-metal type deployment. If the request is directed to a webservice running in Kubernetes, the request now reaches haproxy in (2), which knows the cluster nodes that are available for Ingress. The original 80/TCP packet is now translated to 30000/TCP; this is the TCP port we use internally for the Ingress traffic. This haproxy instance provides load-balancing also for the Kubernetes API as well, using 6443/TCP. It s worth mentioning that unlike the Ingress, the API is only reachable from within the cluster and not from the internet. We have a NGINX-Ingress NodePort service listening in 30000/TCP in every Kubernetes worker node in (3); this helps the request to eventually reach the actual NGINX-Ingress pod in (4), which is listening in 8080/TCP. You can see in the diagram how in the API server (5) we hook the Ingress admission controller (6) to validate Kubernetes Ingress configuration objects before allowing them in for processing by NGINX-Ingress (7). The NGINX-Ingress process knows which tools webservices are online and how to contact them by means of an intermediate Service object in (8). This last Service object means the request finally reaches the actual tool pod in (9). At this point, it is worth noting that our Kubernetes cluster uses internally kube-proxy and Calico, both using Netfilter components to handle traffic. tools-webservice Most user-facing operations are simplified by means of another custom piece of Python code: tools-webservice. This package provides users with the webservice command line utility in our shell bastion hosts. Typical usage is to just run webservice start stop status. This utility creates all the required Kubernetes objects on-demand like Deployment, ReplicaSet, Ingress and Service to ease deploying web apps in Toolforge. Of course, advanced users can interact directly with Kubernetes API and create their custom configuration objects. This utility is just a wrapper, a shortcut. tool-fourohfour and tool-k8s-status The last couple of custom components we would like to mention are the tool-fourohfour and tool-k8s-status web services. These two utilities run inside the cluster as if they were any other user-created tool. The fourohfour tool allows for a controlled handling of HTTP 404 errors, and it works as the default NGINX-Ingress backend. The k8s-status tool shows plenty of information about the cluster itself and each tool running in the cluster, including links to the Server Admin Log, an auto-generated grafana dashboard for metrics, and more. For metrics, we use an external Prometheus server that contacts the Kubernetes cluster to scrape metrics. We created a custom metrics namespace in which we deploy all the different components we use to observe the behavior of the system:

• metrics-server: used by some utilities like kubectl top.
• kube-state-metrics: provides advanced metrics about the state of the cluster.
• cadvisor: to obtain fine-grained metrics about pods, deployments, nodes, etc.

All the Prometheus data we collect is used in several different Grafana dashboards, some of them directed for user information like the ones linked by the k8s-status tool and some others for internal use by us the engineers. These are for internal use but are still public, like the Ingress specific dashboard, or the cluster state dashboard. Working publicly, in a transparent way, is key for the success of CloudVPS in general and Toolforge in particular. Like we commented in the previous post, all the engineering work that was done here was shared by community members. By the community, for the community We think this post sheds some light on how the Toolforge Kubernetes service works, and we hope it could inspire others when trying to build similar services or, even better, help us improve Toolforge itself. Since this was first put into production some months ago we detected already some margin for improvement in a couple of the components. As in many other engineering products, we will follow an iterative approach for evolving the service. Mind that Toolforge is maintained by the Wikimedia Foundation, but you can think of it as a service by the community for the community. We will keep an eye on it and have a list of feature requests and things to improve in the future. We are looking forward to it! This post was originally published in the Wikimedia Tech blog, and is authored by Arturo Borrero Gonzalez and Brooke Storm.

This comic was based on this essay from Augusten Burroughs: How to live unhappily ever after. In addition to the essay, I highly recommend reading his books. It's also been described in psychology as flow.

With full English subtitles

Confict where it comes from and how to deal with it

Communication skills

Other languages

Distributed teams are where people you work with aren t physically co-located, ie. they re at another office building, home or an outsourced company abroad. They re becoming increasingly popular, for DevOps and other teams, due to recruitment, diversity, flexibility and cost savings. Challenges arise due to timezones, language barriers, cultures and ways of working. People actively participating in Open Source communities tend to be effective in distributed teams. This session looks at how to apply core Open Source principles to distributed teams in Enterprise organisations, and the importance of shared purposes/goals, (mis)communication, leading vs managing teams, sharing and learning. We'll also look at practical aspects of what's worked well for others, such as alternatives to daily standups, promoting video conferencing, time management and virtual coffee breaks. This session is relevant for those leading or working in distributed teams, wanting to know how to cultivate an inclusive culture of increased trust and collaboration that leads to increased productivity and performance.

Because of the lock-down in France and thanks to Lucas, I have been able to make some progress rebuilding Debian with clang instead of gcc.

TLDR Instead of patching clang itself, I used a different approach this time: patching Debian tools or implementing some workaround to mitigate an issue.
The percentage of packages failing drop from 4.5% to 3.6% (1400 packages to 1110 - on a total of 31014). I focused on two classes of issues:

• The qmake bug - 250 issues - https://clang.debian.net/status.php?version=9.0.1&key=FAILED_PARSE_DEFAULT
  Basically, Qmake refuses to build a package when gcc's binary is replaced by clang

• Different symbols generated in the libraries - 137 issues - https://clang.debian.net/status.php?version=9.0.1&key=CHANGE_SYM_LIB
  Clang and gcc differ in term of symbols generated in the libraries. While there isn't any impact on the execution or linking, the Debian tooling doesn't have the capability to manage the two cases.
  For more details, see https://bugs.llvm.org/show_bug.cgi?id=30441 & https://bugs.llvm.org/show_bug.cgi?id=45322.

Qmake As I have no intention to merge the patch upstream, I used a very dirty workaround. I overwrote the g++ qmake file by clang's:
https://salsa.debian.org/lucas/collab-qa-tools/-/blob/master/modes/clang10#L44-47 I dropped the number of this failure to 0, making some packages build flawlessly (example: qtcreator, chessx, fwbuilder, etc). However, some packages are still failing later and therefore increasing the number of failures in some other categories like link error. For example, qtads fails because of ordered comparison between pointer and zero or oscar fails on a -Werror,-Wdeprecated-copy error. Breaking the build later also highlighted some new classes of issues which didn't occur with clang < 10.
For example, warnings related to C++ range loop or implicit int float conversion (I fixed a bunch of them in Firefox) .

Symbol differences Historically, symbol management for C++ in Debian has been a pain. Russ Allbery wrote a blog post in 2012 explaining the situation. AFAIK, it hasn't changed much.
Once more, I took the dirty approach: if there new or missing symbols, don't fail the build.
The rational is the following: Packages in the Debian archive are supposed to build without any issue. If there is new or missing symbols, it is probably clang generating a different library but this library is very likely working as expected (and usable by a program compiled with g++ or clang). It is purely a different approach taken by the compiler developer. In order to mitigate this issue, before the build starts, I am modifying dpkg-gensymbols to transform the error into a warning.
So, the typical Debian error some new symbols appeared in the symbols file or some symbols or patterns disappeared in the symbols file will NOT fail the build. Unsurprisingly, all but one package (libktorrent) build. Even if I am pessimistic, I reported a bug on dpkg-dev to evaluate if we could improve dpkg-gensymbol not to fail on these cases.

Next steps The next offender is Imake.tmpl:2243:10: fatal error: ' X11 .rules' file not found with more than an hundred occurrences, reported upstream quite sometime ago. Then, the big issues are going to be much harder to fix as they are real issues/warnings (with -Werror) in the code of the packages. Example: -Wc++11-narrowing & Wreserved-user-defined-literal... The list is long.
I will probably work on that when llvm/clang 11 are in RC phase.

For maintainers & upstream Maintainer of Debian/Ubuntu packages? I am providing a list of failing packages per maintainer: https://clang.debian.net/maintainers.php
For upstream, it is also easy to test with clang. Usually, apt install clang && CC=clang CXX=clang++ <build step> is good enough.

Conclusion With these two changes, I have been able to fix about 290 packages. I think I will be able to get that down a bit more but we will soon reach a plateau as many warnings/issues will have to fix in the C/C++ code itself.

Paul-F lix Armand-Delille (3 July 1874 in Fourchambault, Ni vre 4 September 1963) was a physician, bacteriologist, professor, and member of the French Academy of Medicine who accidentally brought about the collapse of rabbit populations throughout much of Europe and beyond in the 1950s by infecting them with myxomatosis.

Charles Franklin Kettering (August 29, 1876 November 25, 1958) sometimes known as Charles "Boss" Kettering[1] was an American inventor, engineer, businessman, and the holder of 186 patents.[2] He was a founder of Delco, and was head of research at General Motors from 1920 to 1947. Among his most widely used automotive developments were the electrical starting motor[3] and leaded gasoline.[4][5] In association with the DuPont Chemical Company, he was also responsible for the invention of Freon refrigerant for refrigeration and air conditioning systems. At DuPont he also was responsible for the development of Duco lacquers and enamels, the first practical colored paints for mass-produced automobiles. While working with the Dayton-Wright Company he developed the "Bug" aerial torpedo, considered the world's first aerial missile.[6] He led the advancement of practical, lightweight two-stroke diesel engines, revolutionizing the locomotive and heavy equipment industries. In 1927, he founded the Kettering Foundation, a non-partisan research foundation. He was featured on the cover of Time magazine on January 9, 1933.

John Charles Cutler (June 29, 1915 February 8, 2003) was a senior surgeon, and the acting chief of the venereal disease program in the United States Public Health Service. After his death, his involvement in several controversial and unethical medical studies of syphilis was revealed, including the Guatemala and the Tuskegee syphilis experiments.

Ivy Ledbetter Lee (July 16, 1877 November 9, 1934) was an American publicity expert and a founder of modern public relations. Lee is best known for his public relations work with the Rockefeller family. His first major client was the Pennsylvania Railroad, followed by numerous major railroads such as the New York Central, the Baltimore and Ohio, and the Harriman lines such as the Union Pacific. He established the Association of Railroad Executives, which included providing public relations services to the industry. Lee advised major industrial corporations, including steel, automobile, tobacco, meat packing, and rubber, as well as public utilities, banks, and even foreign governments. Lee pioneered the use of internal magazines to maintain employee morale, as well as management newsletters, stockholder reports, and news releases to the media. He did a great deal of pro bono work, which he knew was important to his own public image, and during World War I, he became the publicity director for the American Red Cross.[1]

Like each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In April, 284.5 work hours have been dispatched among 14 paid contributors. Their reports are available:

• Abhijith PA did 10.0h (out of 14h assigned), thus carrying over 4h to May.
• Adrian Bunk did nothing (out of 28.75h assigned), thus is carrying over 28.75h for May.
• Ben Hutchings did 26h (out of 20h assigned and 8.5h from March), thus carrying over 2.5h to May.
• Brian May did 10h (out of 10h assigned).
• Chris Lamb did 18h (out of 18h assigned).
• Dylan A ssi did 6h (out of 6h assigned).
• Emilio Pozuelo Monfort did not report back about their work so we assume they did nothing (out of 28.75h assigned plus 17.25h from March), thus is carrying over 46h for May.
• Markus Koschany did 11.5h (out of 28.75h assigned and 38.75h from March), thus carrying over 56h to May.
• Mike Gabriel did 1.5h (out of 8h assigned), thus carrying over 6.5h to May.
• Ola Lundqvist did 13.5h (out of 12h assigned and 8.5h from March), thus carrying over 7h to May.
• Roberto C. S nchez did 28.75h (out of 28.75h assigned).
• Sylvain Beucler did 28.75h (out of 28.75h assigned).
• Thorsten Alteholz did 28.75h (out of 28.75h assigned).
• Utkarsh Gupta did 24h (out of 24h assigned).

Evolution of the situation In April we dispatched more hours than ever and another was new too, we had our first (virtual) contributors meeting on IRC! Logs and minutes are available and we plan to continue doing IRC meetings every other month.
Sadly one contributor decided to go inactive in April, Hugo Lefeuvre.
Finally, we like to remind you, that the end of Jessie LTS is coming in less than two months!
In case you missed it (or missed to act), please read this post about keeping Debian 8 Jessie alive for longer than 5 years. If you expect to have Debian 8 servers/devices running after June 30th 2020, and would like to have security updates for them, please get in touch with Freexian. The security tracker currently lists 4 packages with a known CVE and the dla-needed.txt file has 25 packages needing an update. Thanks to our sponsors New sponsors are in bold.

• Platinum sponsors:
• TOSHIBA (for 56 months)
• GitHub (for 46 months)
• Civil Infrastructure Platform (CIP) (for 24 months)
• Gold sponsors:
• Blablacar (for 71 months)
• Roche Diagnostics International AG (for 67 months)
• Linode (for 61 months)
• Babiel GmbH (for 50 months)
• Plat Home (for 49 months)
• University of Oxford (for 6 months)
• Silver sponsors:
• The Positive Internet Company (for 72 months)
• Domeneshop AS (for 71 months)
• Nantes M tropole (for 65 months)
• Univention GmbH (for 57 months)
• Universit Jean Monnet de St Etienne (for 57 months)
• Ribbon Communications, Inc. (for 51 months)
• Exonet B.V. (for 40 months)
• Leibniz Rechenzentrum (for 34 months)
• CINECA (for 24 months)
• Minist re de l Europe et des Affaires trang res (for 18 months)
• Cloudways Ltd (for 7 months)
• Dinahosting SL (for 5 months)
• Bronze sponsors:
• Seznam.cz, a.s. (for 72 months)
• Evolix (for 71 months)
• MyTux (for 71 months)
• Linuxhotel GmbH (for 69 months)
• Intevation GmbH (for 68 months)
• Daevel SARL (for 67 months)
• Bitfolk LTD (for 66 months)
• Megaspace Internet Services GmbH (for 66 months)
• Greenbone Networks GmbH (for 65 months)
• NUMLOG (for 65 months)
• WinGo AG (for 64 months)
• Ecole Centrale de Nantes LHEEA (for 61 months)
• Sig-I/O (for 58 months)
• Entr ouvert (for 56 months)
• Adfinis SyGroup AG (for 53 months)
• Laboratoire LEGI UMR 5519 / CNRS (for 48 months)
• Tesorion (for 48 months)
• GNI MEDIA (for 47 months)
• Bearstech (for 39 months)
• LiHAS (for 39 months)
• People Doc (for 35 months)
• Catalyst IT Ltd (for 33 months)
• Supagro (for 29 months)
• Demarcq SAS (for 27 months)
• Universit Grenoble Alpes (for 14 months)
• TouchWeb SAS (for 6 months)
• SPiN AG

No comment Liked this article? Click here. My blog is Flattr-enabled.

Last night, Lars, Mark, and I discussed Jeremy Kun's The communicative value of using Git well post. While a lot of our discussion was spawned by the article, we did go off-piste a little, and I hope that my notes below will enlighten you all as to a bit of how we see revision control these days. It was remarkably pleasant to read an article where the comments section wasn't a cesspool of horror, so if this posting encourages you to go and read the article, don't stop when you reach the bottom -- the comments are good and useful too.

---

This was a fairly non-contentious article for us though each of us had points we wished to bring up and chat about it turned into a very convivial chat. We saw the main thrust of the article as being about using the metadata of revision control to communicate intent, process, and decision making. We agreed that it must be possible to do so effectively with Mercurial (thus deciding that the mention of it was simply a bit of clickbait / red herring) and indeed Mark figured that he was doing at least some of this kind of thing way back with CVS. We all discussed how knowing the fundamentals of Git's data model improved our ability to work wih the tool. Lars and I mentioned how jarring it has originally been to come to Git from revision control systems such as Bazaar (bzr) but how over time we came to appreciate Git for what it is. For Mark this was less painful because he came to Git early enough that there was little more than the fundamental data model, without much of the porcelain which now exists. One point which we all, though Mark in particular, felt was worth considering was that of distinguishing between published and unpublished changes. The article touches on it a little, but one of the benefits of the workflow which Jeremy espouses is that of using the revision control system as an integral part of the review pipeline. This is perhaps done very well with Git based workflows, but can be done with other VCSs. With respect to the points Jeremy makes regarding making commits which are good for reviewing, we had a general agreement that things were good and sensible, to a point, but that some things were missed out on. For example, I raised that commit messages often need to be more thorough than one-liners, but Jeremy's examples (perhaps through expedience for the article?) were all pretty trite one-liners which perhaps would have been entirely obvious from the commit content. Jeremy makes the point that large changes are hard to review, and Lars poined out that Greg Wilson did research in this area, and at least one article mentions 200 lines as a maximum size of a reviewable segment. I had a brief warble at this point about how reviewing needs to be able to consider the whole of the intended change (i.e. a diff from base to tip) not just individual commits, which is also missing from Jeremy's article, but that such a review does not need to necessarily be thorough and detailed since the commit-by-commit review remains necessary. I use that high level diff as a way to get a feel for the full shape of the intended change, a look at the end-game if you will, before I read the story of how someone got to it. As an aside at this point, I talked about how Jeremy included a 'style fixes' commit in his example, but I loathe seeing such things and would much rather it was either not in the series because it's unrelated to it; or else the style fixes were folded into the commits they were related to. We discussed how style rules, as well as commit-bisectability, and other rules which may exist for a codebase, the adherence to which would form part of the checks that a code reviewer may perform, are there to be held to when they help the project, and to be broken when they are in the way of good communication between humans. In this, Lars talked about how revision control histories provide high level valuable communication between developers. Communication between humans is fraught with error and the rules are not always clear on what will work and what won't, since this depends on the communicators, the context, etc. However whatever communication rules are in place should be followed. We often say that it takes two people to communicate information, but when you're writing commit messages or arranging your commit history, the second party is often nebulous "other" and so the code reviewer fulfils that role to concretise it for the purpose of communication. At this point, I wondered a little about what value there might be (if any) in maintaining the metachanges (pull request info, mailing list discussions, etc) for historical context of decision making. Mark suggested that this is useful for design decisions etc but not for the style/correctness discussions which often form a large section of review feedback. Maybe some of the metachange tracking is done automatically by the review causing the augmentation of the changes (e.g. by comments, or inclusion of design documentation changes) to explain why changes are made. We discussed how the "rebase always vs. rebase never" feeling flip-flopped in us for many years until, as an example, what finally won Lars over was that he wants the history of the project to tell the story, in the git commits, of how the software has changed and evolved in an intentional manner. Lars said that he doesn't care about the meanderings, but rather about a clear story which can be followed and understood. I described this as the switch from "the revision history is about what I did to achieve the goal" to being more "the revision history is how I would hope someone else would have done this". Mark further refined that to "The revision history of a project tells the story of how the project, as a whole, chose to perform its sequence of evolution." We discussed how project history must necessarily then contain issue tracking, mailing list discussions, wikis, etc. There are exist free software projects where part of their history is forever lost because, for example, the project moved from Sourceforge to Github, but made no effort (or was unable) to migrate issues or somesuch. Linkages between changes and the issues they relate to can easily be broken, though at least with mailing lists you can often rewrite URLs if you have something consistent like a Message-Id. We talked about how cover notes, pull request messages, etc. can thus also be lost to some extent. Is this an argument to always use merges whose message bodies contain those details, rather than always fast-forwarding? Or is it a reason to encapsulate all those discussions into git objects which can be forever associated with the changes in the DAG? We then diverted into discussion of CI, testing every commit, and the benefits and limitations of automated testing vs. manual testing; though I think that's a little too off-piste for even this summary. We also talked about how commit message audiences include software perhaps, with the recent movement toward conventional commits and how, with respect to commit messages for machine readability, it can become very complex/tricky to craft good commit messages once there are multiple disparate audiences. For projects the size of the Linux kernel this kind of thing would be nearly impossible, but for smaller projects, perhaps there's value. Finally, we all agreed that we liked the quote at the end of the article, and so I'd like to close out by repeating it for you all... Hal Abelson famously said:

Programs must be written for people to read, and only incidentally for machines to execute.

Jeremy agrees, as do we, and extends that to the metacommit information as well.

Welcome to the April 2020 report from the Reproducible Builds project. In our regular reports we outline the most important things that we and the rest of the community have been up to over the past month. What are reproducible builds? One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. But whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into seemingly secure software during the various compilation and distribution processes.

News It was discovered that more than 725 malicious packages were downloaded thousands of times from RubyGems, the official channel for distributing code for the Ruby programming language. Attackers used a variation of typosquatting and replaced hyphens and underscores (for example, uploading a malevolent atlas-client in place of atlas_client) that executed a script that intercepted Bitcoin payments. (Ars Technica report) Bernhard M. Wiedemann launched ismypackagereproducibleyet.org, a service that takes a package name as input and displays whether the package is reproducible in a number of distributions. For example, it can quickly show the status of Perl as being reproducible on openSUSE but not in Debian. Bernhard also improved the documentation of his unreproducible package to add some example patches for hash issues. [ ]. There was a post on Chaos Computer Club s website listing Ten requirements for the evaluation of Contact Tracing apps in relation to the SARS-CoV-2 epidemic. In particular:

4. Transparency and verifiability: The complete source code for the app and infrastructure must be freely available without access restrictions to allow audits by all interested parties. Reproducible build techniques must be used to ensure that users can verify that the app they download has been built from the audited source code.

Elsewhere, Nicolas Boulenguez wrote a patch for the Ada programming language component of the GCC compiler to skip -f.*-prefix-map options when writing Ada Library Information files. Amongst other properties, these .ali files embed the compiler flags used at the time of the build which results in the absolute build path being recorded via -ffile-prefix-map, -fdebug-prefix-map, etc. In the Arch Linux project, kpcyrd reported that they held their first rebuilder workshop . The session was held on IRC and participants were provided a document with instructions on how to install and use Arch s repro tool. The meeting resulted in multiple people with no prior experience of Reproducible Builds validate their first package. Later in the month he also announced that it was now possible to run independent rebuilders under Arch in a hands-off, everything just works solution to distributed package verification. Mathias Lang submitted a pull request against dmd, the canonical compiler for the D programming languageto add support for our SOURCE_DATE_EPOCH environment variable as well the other C preprocessor tokens such __DATE__, __TIME__ and __TIMESTAMP__ which was subsequently merged. SOURCE_DATE_EPOCH defines a distribution-agnostic standard for build toolchains to consume and emit timestamps in situations where they are deemed to be necessary. [ ] The Telegram instant-messaging platform announced that they had updated to version 5.1.1 continuing their claim that they are reproducible according to their full instructions and therefore verifying that its original source code is exactly the same code that is used to build the versions available on the Apple App Store and Google Play distribution platforms respectfully. Lastly, Herv Boutemy reported that 97% of the current development versions of various Maven packages appear to have a reproducible build. [ ]

Distribution work In Debian this month, 89 reviews of Debian packages were added, 21 were updated and 33 were removed this month adding to our knowledge about identified issues. Many issue types were noticed, categorised and updated by Chris Lamb, including:

• captures_build_path_in_hd5_database_files
• cargo_installs_crates2_json
• nondeterministic_devhelp_documentation_generated_by_gtk_doc
• ros_dynamic_reconfigure_captures_build_path

In addition, Holger Levsen filed a feature request against debrebuild, a tool for rebuilding a Debian package given a .buildinfo file, proposing to add --standalone or --one-shot-mode functionality.
In openSUSE, Bernhard M. Wiedemann made the following changes:

• blender (sort C readdir call, rejected upstream)
• guile/guix (parallelism race condition)
• mingw32-filesystem/mingw32-binutils (sort readdir, filesystem, toolchain)
• mingw64-filesystem/mingw64-binutils (sort readdir, filesystem, toolchain)
• musescore (non-deterministic .zip files)
• OBS (FTBFS in rebuild)
• perl-Image-Sane (report hung build on a single core VM)
• ruby2.7 (date, already upstream)
• vtk (drop unreproducible .pyc file)

In Arch Linux, a rebuilder instance has been setup at reproducible.archlinux.org that is rebuilding Arch s [core] repository directly. The first rebuild has led to approximately 90% packages reproducible contrasting with 94% on the Reproducible Build s project own ArchLinux status page on tests.reproducible-builds.org that continiously builds packages and does not verify Arch Linux packages. More information may be found on the corresponding wiki page and the underlying decisions were explained on our mailing list.

Software development

diffoscope Chris Lamb made the following changes to diffoscope, the Reproducible Builds project s in-depth and content-aware diff utility that can locate and diagnose reproducibility issues (including preparing and uploading versions 139, 140, 141, 142 and 143 to Debian which were subsequently uploaded to the backports repository):

• Comparison improvements:
  • Dalvik .dex files can also serve as APK containers so restrict the narrower identification of .dex files to files ending with this extension and widen the identification of APK files to when file(1) discovers a Dalvik file. (#28)
  • Add support for Hierarchical Data Format (HD5) files. (#95)
  • Add support for .p7c and .p7b certificates. (#94)
  • Strip paths from the output of zipinfo(1) warnings. (#97)
  • Don t uselessly include the JSON similarity percentage if it is 0.0% . [ ]
  • Render multi-line difference comments in a way to show indentation. (#101)
• Testsuite improvements:
  • Add pdftotext as a requirement to run the PDF test_metadata text. (#99)
  • apktool 2.5.0 changed the handling of output of XML schemas so update and restrict the corresponding test to match. (#96)
  • Explicitly list python3-h5py in debian/tests/control.in to ensure that we have this module installed during a test run to generate the fixtures in these tests. [ ]
  • Correct parsing of ./setup.py test --pytest-args arguments. [ ]
• Misc:
  • Capitalise Ordering differences only in text comparison comments. [ ]
  • Improve documentation of FILE_TYPE_HEADER_PREFIX and FALLBACK_FILE_TYPE_HEADER_PREFIX to highlight that only the first 16 bytes are used. [ ]

Michael Osipov created a well-researched merge request to return diffoscope to using zipinfo directly instead of piping input via /dev/stdin in order to ensure portability to the BSD operating system [ ]. In addition, Ben Hutchings documented how --exclude arguments are matched against filenames [ ] and Jelle van der Waa updated the LLVM test fixture difference for LLVM version 10 [ ] as well as adding a reference to the name of the h5dump tool in Arch Linux [ ]. Lastly, Mattia Rizzolo also fixed in incorrect build dependency [ ] and Vagrant Cascadian enabled diffoscope to locate the openssl and h5dump packages on GNU Guix [ ][ ], and updated diffoscope in GNU Guix to version 141 [ ] and 143 [ ].

strip-nondeterminism strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. In April, Chris Lamb made the following changes:

• Add deprecation plans to all handlers documenting how or if they could be disabled and eventually removed, etc. (#3)
• Normalise *.sym files as Java archives. (#15)
• Add support for custom .zip filename filtering and exclude two patterns of files generated by Maven projects in fork mode. (#13)

disorderfs disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues. This month, Chris Lamb fixed a long-standing issue by not drop UNIX groups in FUSE multi-user mode when we are not root (#1) and uploaded version 0.5.9-1 to Debian unstable. Vagrant Cascadian subsequently refreshed disorderfs in GNU Guix to version 0.5.9 [ ].

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

• Bernhard M. Wiedemann:
  • elixir (parallelism)
  • gnutls (build failure)
  • moonjit/bcc (compile-time CPU-detection)
  • openstack (backport of patch to drop unreproducible sphinx .pickle files)
  • x3270 (merged, update date patch)
• Chris Lamb:
  • #958301 filed against dh-cargo.
  • #956549 filed against gmap.
  • #956591 filed against gpick.
  • #956477 filed against herbstluftwm.
  • #956304 filed against libcamera.
  • #956589 filed against libctl.
  • #956408 filed against minetest-mod-xdecor.
  • #955783 filed against netgen-lvs.
  • #958110 filed against nickle.
  • #958381 filed against nmrpflash.
  • #958382 filed against node-mqtt.
  • #956473 filed against sprai.
  • #955501 filed against yaz.
  • #956583 filed against xxhash.

In addition, Bernhard informed the following projects that their packages are not reproducible:

• acoular (report unknown non-determinism)
• cri-o (report a date issue)
• gnutls (report certtool being unable to extend certificates beyond 2049)
• gnutls (report copyright year variation)
• libxslt (report a bug about non-deterministic output from data corruption)
• python-astropy (report a future build failure in 2021)

Project documentation This month, Chris Lamb made a large number of changes to our website and documentation in the following categories:

• Community engagement improvements:
  • Update instructions to register for Salsa on our Contribute page now that the signup process has been overhauled. [ ]
  • Make it clearer that joining the rb-general mailing list is probably a first step for contributors to take. [ ]
  • Make our full contact information easier to find in the footer (#19) and improve text layout using bullets to separate sections [ ].
• Accessibility:
  • To improve accessibility, make all links underlined. (#12)
  • Use an enhanced foreground/background contrast ratio of 7.04:1. (#11)
• General improvements:
  • Add a new Academic publications page. (#22)
  • Add Trezor to our list of affiliated projects. (#26)
  • Add the JVM page to the documentation index (#17) and tidy the page itself a little [ ].
  • Add a GNU Libtool pointer to the Archive metadata documentation page. [ ]
• Internals:
  • Move to using jekyll-redirect-from over manual redirect pages [ ][ ] and add a redirect from /docs/buildinfo/ to /docs/recording/. (#23)
  • Limit the website self-check to not scan generated files [ ] and remove the old layout checker now that I have migrated all them [ ].
  • Move the news archive under the /news/ namespace [ ] and improve formatting of archived news links [ ].
  • Various improvements to the draft template generation. [ ][ ][ ][ ]

In addition, Holger Levsen clarified exactly which month we ceased to do weekly reports [ ] and Mattia Rizzolo adjusted the title style of an event page [ ]. Marcus Hoffman also started a discussion on our website s issue tracker asking for clarification on embedded signatures and Chris Lamb subsequently replied and asked Marcus to go ahead and propose a concrete change.

Testing framework We operate a large and many-featured Jenkins-based testing framework that powers tests.reproducible-builds.org that, amongst many other tasks, tracks the status of our reproducibility efforts as well as identifies any regressions that have been introduced.

• Chris Lamb:
  • Print the build environment prior to executing a build. [ ]
  • Drop a misleading disorderfs-debug prefix in log output when we change non-disorderfs things in the file and, as it happens, do not run disorderfs at all. [ ]
  • The CSS for the package report pages added a margin to all <a> HTML elements under <li> ones, which was causing a comma/bullet spacing issue. [ ]
  • Tidy the copy in the project links sidebar. [ ]
• Holger Levsen:
  • General:
    • Install jekyll-redirect-from as it now needed by the reproducible-builds.org website. [ ]
    • Improve/correct log parsing rules. [ ][ ]
  • Debian:
    • Reduce scheduling frequency of the buster distribution on the arm64 architecture, etc.. [ ][ ]
    • Show builds per day on a per-architecture basis for the last year on the Debian dashboard. [ ]
    • Drop the Subgraph OS package set as development halted in 2017 or 2018. [ ]
    • Update debrebuild to version from the latest version of devscripts. [ ][ ]
    • Add or improve various parts of the documentation. [ ][ ][ ]
  • Work on a Debian rebuilder:
    • Integrate sbuild. [ ][ ][ ][ ][ ]
    • Select a random .buildinfo file and attempt to build and compare the result. [ ][ ][ ][ ]
    • Improve output and related output formatting. [ ][ ][ ][ ][ ]
    • Outline next steps for the development of the tool. [ ][ ][ ]
    • Various refactoring and code improvements. [ ][ ][ ]

Lastly, Mattia Rizzolo fixed some log parsing code regarding potentially-harmless warnings from package installation [ ][ ] and the usual build node maintenance was performed by Holger Levsen [ ][ ][ ] and Mattia Rizzolo [ ][ ][ ].

---

Misc news On our mailing list this month, Santiago Torres asked whether we were still publishing releases of our tools to our website and Chris Lamb replied that this was not the case and fixed the issue. Later in the month Santiago also reported that the signature for the disorderfs package did not pass its GPG verification which was also fixed by Chris Lamb. Hans-Christoph Steiner of the Guardian Project asked whether there would be interest in making our website translatable which resulted in a WIP merge request being filed against the website and a discussion on how to track translation updates.
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.
• Twitter: @ReproBuilds @reproducible_builds@fosstodon.org
• Reddit: /r/ReproducibleBuilds
• Mailing list: rb-general@lists.reproducible-builds.org

This month s report was written by Bernhard M. Wiedemann, Chris Lamb, Daniel Shahaf, Holger Levsen, Jelle van der Waa, kpcyrd, Mattia Rizzolo and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

Currently there is political debate about when businesses should be reopened after the Covid19 quarantine. Small Businesses One argument for reopening things is for the benefit of small businesses. The first thing to note is that the protests in the US say I need a haircut not I need to cut people s hair . Small businesses won t benefit from reopening sooner. For every business there is a certain minimum number of customers needed to be profitable. There are many comments from small business owners that want it to remain shutdown. When the government has declared a shutdown and paused rent payments and provided social security to employees who aren t working the small business can avoid bankruptcy. If they suddenly have to pay salaries or make redundancy payouts and have to pay rent while they can t make a profit due to customers staying home they will go bankrupt. Many restaurants and cafes make little or no profit at most times of the week (I used to be 1/3 owner of an Internet cafe and know this well). For such a company to be viable you have to be open most of the time so customers can expect you to be open. Generally you don t keep a cafe open at 3PM to make money at 3PM, you keep it open so people can rely on there being a cafe open there, someone who buys a can of soda at 3PM one day might come back for lunch at 1:30PM the next day because they know you are open. A large portion of the opening hours of a most retail companies can be considered as either advertising for trade at the profitable hours or as loss making times that you can t close because you can t send an employee home for an hour. If you have seating for 28 people (as my cafe did) then for about half the opening hours you will probably have 2 or fewer customers in there at any time, for about a quarter the opening hours you probably won t cover the salary of the one person on duty. The weekend is when you make the real money, especially Friday and Saturday nights when you sometimes get all the seats full and people coming in for takeaway coffee and snacks. On Friday and Saturday nights the 60 seat restaurant next door to my cafe used to tell customers that my cafe made better coffee. It wasn t economical for them to have a table full for an hour while they sell a few cups of coffee, they wanted customers to leave after dessert and free the table for someone who wants a meal with wine (alcohol is the real profit for many restaurants). The plans of reopening with social distancing means that a 28 seat cafe can only have 14 chairs or less (some plans have 25% capacity which would mean 7 people maximum). That means decreasing the revenue of the most profitable times by 50% to 75% while also not decreasing the operating costs much. A small cafe has 2-3 staff when it s crowded so there s no possibility of reducing staff by 75% when reducing the revenue by 75%. My Internet cafe would have closed immediately if forced to operate in the proposed social distancing model. It would have been 1/4 of the trade and about 1/8 of the profit at the most profitable times, even if enough customers are prepared to visit and social distancing would kill the atmosphere. Most small businesses are barely profitable anyway, most small businesses don t last 4 years in normal economic circumstances. This reopen movement is about cutting unemployment benefits not about helping small business owners. Destroying small businesses is also good for big corporations, kill the small cafes and restaurants and McDonald s and Starbucks will win. I think this is part of the motivation behind the astroturf campaign for reopening businesses. Forbes has an article about this [1]. Psychological Issues Some people claim that we should reopen businesses to help people who have psychological problems from isolation, to help victims of domestic violence who are trapped at home, to stop older people being unemployed for the rest of their lives, etc. Here is one article with advice for policy makers from domestic violence experts [2]. One thing it mentions is that the primary US federal government program to deal with family violence had a budget of $130M in 2013. The main thing that should be done about family violence is to make it a priority at all times (not just when it can be a reason for avoiding other issues) and allocate some serious budget to it. An agency that deals with problems that affect families and only has a budget of $1 per family per year isn t going to be able to do much. There are ongoing issues of people stuck at home for various reasons. We could work on better public transport to help people who can t drive. We could work on better healthcare to help some of the people who can t leave home due to health problems. We could have more budget for carers to help people who can t leave home without assistance. Wanting to reopen restaurants because some people feel isolated is ignoring the fact that social isolation is a long term ongoing issue for many people, and that many of the people who are affected can t even afford to eat at a restaurant! Employment discrimination against people in the 50+ age range is an ongoing thing, many people in that age range know that if they lose their job and can t immediately find another they will be unemployed for the rest of their lives. Reopening small businesses won t help that, businesses running at low capacity will have to lay people off and it will probably be the older people. Also the unemployment system doesn t deal well with part time work. The Australian system (which I think is similar to most systems in this regard) reduces the unemployment benefits by $0.50 for every dollar that is earned in part time work, that effectively puts people who are doing part time work because they can t get a full-time job in the highest tax bracket! If someone is going to pay for transport to get to work, work a few hours, then get half the money they earned deducted from unemployment benefits it hardly makes it worthwhile to work. While the exact health impacts of Covid19 aren t well known at this stage it seems very clear that older people are disproportionately affected, so forcing older people to go back to work before there is a vaccine isn t going to help them. When it comes to these discussions I think we should be very suspicious of people who raise issues they haven t previously shown interest in. If the discussion of reopening businesses seems to be someone s first interest in the issues of mental health, social security, etc then they probably aren t that concerned about such issues. I believe that we should have a Universal Basic Income [3]. I believe that we need to provide better mental health care and challenge the gender ideas that hurt men and cause men to hurt women [4]. I believe that we have significant ongoing problems with inequality not small short term issues [5]. I don t think that any of these issues require specific changes to our approach to preventing the transmission of disease. I also think that we can address multiple issues at the same time, so it is possible for the government to devote more resources to addressing unemployment, family violence, etc while also dealing with a pandemic.

• [1] http://tinyurl.com/ydc96see
• [2] http://tinyurl.com/ycvy65re
• [3] https://etbe.coker.com.au/2013/12/08/basic-income-australia/
• [4] https://etbe.coker.com.au/2019/01/13/are-men-the-victims/
• [5] https://etbe.coker.com.au/2011/12/09/some-ows-stuff/

The way developers somehow think DevOps is (or should be) an abbreviation of "Developers doing/replacing Operations" is terrifying to me.I'm also in the same boat as the author, in that I recommend and target Debian Stable + Backports (and some vendor/community repos when required).

However hard you work on documentation, it won't work for your software - unless you do it the right way.

I'm going to preach the wonders of Python dataclasses, but for reasons of interested to those who have already gone down the delightful rabbit-hole of typed Python. So let me start with a quick plug for mypy if you haven't heard about it.

Subscribe in a reader Subscribe by Email

Database of 15500 abandonware games free. One of the most complete video games museum. Take a trip down Memory Lane now! Warning: whole weekends can be lost.

Changes

• apt-offline: fix typos
• myrepos: cleanups (1 2), better error messages (1 2)
• Linux kernel: enable dm raid discard support when any devices support discard
• gh: fix .deb install info
• equivs: fix dir message
• webdl: fix iview episode order (sent privately)
• spelling dictionaries: codespell (1 2 3 4 5), lintian (1 2 3 4)
• Debian blends website: fix DDTP URLs
• Debian developers reference: fix DDTP URLs
• Debian bits blog: fix DDTP URLs
• Debian screenshots: deleted scm (manual page)
• Debian package tracking system: add options to disable oldoldstable pockets (1 2)
• Debian website: fix DDTP (1 2) and DebConf sponsors URLs
• Debian userdir-ldap: document #debian-private
• Debian package uploads: purple-discord (1 2 3)
• Debian sysadmin wiki pages: input/howto/postgres
• Debian website: document #debian-private
• Debian wiki pages: apcupsd, Blends/TasksPages, BOINC/Development, configure, DDTP (Migration), DebianAMD64, DebianEdu/Status/Buster, DebianHosting, DebianInstaller/Preseed/EditIso, DebianKernel/UserspaceTools, DebianNetDomains, DebianPartyLine/Mumble, DebianScience/Classification, DebianWannaBuildInfrastructureOnOneServer, DebianWiki/I18n, Derivatives/CensusQA, DevelopmentQuestions, EmbeddedCopies, EtcUnderRevisionControl, Exploits, Firmware/Updates, FlashBIOS, Fonts, Games/Team, Hardware/Wanted, HelpDebian/FAQ, HowToUpgradeKernel, InstallingDebianOn/CompuLab, IRC, Javascript/Policy, KdeMultimedia, L10n/DDTSS, LinuxMagazines, MemberBenefits (1 2), MiscellaneousQuestions, Mobile, OpenStack/OpenStack/todo, PackageArchitectureAlmostAny, PackagesArchSpecific, PageFragments/HowOrganisationsCanHelpDebian, qa.debian.org, ReleaseParty (Bullseye, Buster, SuiteTemplate), Services, Simple-CDD/Bzr, SparcPorting, SSH, SSH, StaticLinking, Teams (DebConf/Bursaries, DebianBuildd, DebianWiki, FrontDesk/DcSiteDevel, I18n), VerifyPAS
• vcs-home wiki pages: index (1 2)
• FOSSjobs wiki pages: resources

Issues

• Crashes in tracker-miner-fs, Xwayland
• Upgrade failure with unattended-upgrades
• Conffile removal needed in rubocop, parcimonie
• Patch backport needed in purple-discord
• Underlinking in bind9-libs
• Features in needrestart, buildd.debian.org, procps (1 2 3), cron, Debian code search, Debian package tracker, pass
• Broken HTML in evolution (sent privately, fixed by upstream)
• Warnings in urwid, webdl (sent privately)
• Removal needed for flasm (1 2)

Review

• Spam: reported 1 Debian bug reports and 178 Debian mailing list posts
• Debian wiki: RecentChanges for the month
• Debian screenshots:
  • approved dablin displaycal enfuse evolvotron flameshot flpsed font-viewer gimp-plugin-registry glue-sprite gpick gpscorrelate gpscorrelate-gui gtkmorph guetzli gummi hdrmerge icc-profiles-free inkscape-tutorials jhead jp2a jpeginfo jpegjudge kde-spectacle libjpeg-turbo-progs magic metacam pianobooster povray-examples printer-driver-cups-pdf propaganda-debian rawtherapee recoverjpeg renrot scm screengrab scrot shanty showfoto simple-image-reducer simple-scan sxiv teeworlds tuxpaint valentina viewnior x4d-icons xcftools
  • rejected musescore (ancient map), fastboot (logo)
  • approved deletion (despite bogus reason) of winwrangler (help output)
  • rejected deletion of blockattack (not spam as claimed), chromium (not deceptive as claimed), file-roller (screenshots of newest interface), jwm/onboard/aajm (spam), xfce4 (not a place to request wallpapers), viewvc (HTML jibberish), fonts-noto-color-emoji/sdl-ball/sddm-theme-breeze/hardinfo (jibberish), gufw (non-English screenshots are allowed), wifite (unsubstantiated claim of false information), mysql-workbench (single word)

Administration

• myrepos: fix the forum
• Debian: restart non-responsive tor daemon, restart processes due to OOM, apply debian.net changes for DD with expired key
• Debian wiki: approve accounts
• Debian QA services: deploy changes, auto-disable oldoldstable pockets

Communication

• Initiate discussion about removing purple-discord from Debian stable
• Respond to queries from Debian users and developers on the mailing lists and IRC

Sponsors The purple-discord work was sponsored by my employer. All other work was done on a volunteer basis.

For the foreseeable future I'm working for myself, as a contractor, here in sunny Helsinki, Finland. My existing contract only requires me to work 1.5-2.0 days a week, meaning my week looks something like this:

• Monday & Tuesday
  • I work for the client.
• Wednesday - Friday
  • I act as a stay-at-home dad.

It does mean that I'm available for work Wednesday-Friday though, in the event I can find projects to work upon, or companies who would be willing to accept my invoices. I think of myself as a sysadmin, but I know all about pipelines, automation, system administration, and coding in C, C++, Perl, Ruby, Golang, Lua, etc. On the off-chance anybody reading this has a need for small projects, services, daemons, or APIs to be implemented then don't hesitate to get in touch. I did manage to fill a few days over the past few weeks completing an online course from Helsinki Open University, Devops with Docker, it is possible I'll find some more courses to complete in the future. (There is an upcoming course Devops with Kubernetes which I'll definitely complete.)

Like each month, here comes a report about the work of paid contributors to Debian LTS. Individual reports In March, 252 work hours have been dispatched among 14 paid contributors. Their reports are available:

• Abhijith PA did 16.0h (out of 14h assigned and 2h from February).
• Ben Hutchings did 12.25h (out of 20h assigned and 0.75h from February), thus carrying over 8.5h to April.
• Brian May did 10h (out of 10h assigned).
• Chris Lamb did 18h (out of 18h assigned).
• Dylan A ssi did 6h (out of 6h assigned).
• Emilio Pozuelo Monfort did 19.5h (out of 30h assigned and 6.75h from February), thus carrying over 17.25h to April.
• Hugo Lefeuvre gave back the 12h he got assigned.
• Markus Koschany did 10h (out of 30h assigned and 18.75h from February), thus carrying over 38.75h to April.
• Mike Gabriel did 10.25h (out of 8h assigned and 2.25h from February).
• Ola Lundqvist did 6h (out of 12h assigned and 2.5h from February), thus carrying over 8.5h to April.
• Roberto C. S nchez did 6.25h (out of 8h assigned), and gave back the remaining 1.75h.
• Sylvain Beucler did 30h (out of 30h assigned).
• Thorsten Alteholz did 30h (out of 30h assigned).
• Utkarsh Gupta did 24h (out of 24h assigned).

Evolution of the situation March was a strange month for many people all over the globe. Here we ll just express our hopes that you are and will be well! LTS gained a new contributor in March, Anton Gladky, however he then decided to become active later this year. Similarly Hugo Lefeuvre notified us that he ll be inactive in April. In case you missed it (or missed to act), please read this post about keeping Debian 8 Jessie alive for longer than 5 years. If you expect to have Debian 8 servers/devices running after June 30th 2020, and would like to have security updates for them, please get in touch with Freexian. Hurry up: the end of Jessie LTS is coming in less than three months! The security tracker currently lists 25 packages with a known CVE and the dla-needed.txt file has 23 packages needing an update. Thanks to our sponsors New sponsors are in bold.

• Platinum sponsors:
• TOSHIBA (for 55 months)
• GitHub (for 45 months)
• Civil Infrastructure Platform (CIP) (for 22 months)
• Gold sponsors:
• Blablacar (for 70 months)
• Roche Diagnostics International AG (for 65 months)
• Linode (for 59 months)
• Babiel GmbH (for 49 months)
• Plat Home (for 48 months)
• University of Oxford (for 4 months)
• Silver sponsors:
• The Positive Internet Company (for 71 months)
• Domeneshop AS (for 70 months)
• Nantes M tropole (for 64 months)
• Univention GmbH (for 56 months)
• Universit Jean Monnet de St Etienne (for 56 months)
• Ribbon Communications, Inc. (for 49 months)
• Exonet B.V. (for 39 months)
• Leibniz Rechenzentrum (for 33 months)
• CINECA (for 23 months)
• Minist re de l Europe et des Affaires trang res (for 17 months)
• Cloudways Ltd (for 6 months)
• Dinahosting SL (for 4 months)
• Bronze sponsors:
• Seznam.cz, a.s. (for 71 months)
• Evolix (for 70 months)
• MyTux (for 70 months)
• Intevation GmbH (for 67 months)
• Linuxhotel GmbH (for 67 months)
• Daevel SARL (for 66 months)
• Bitfolk LTD (for 65 months)
• Megaspace Internet Services GmbH (for 65 months)
• Greenbone Networks GmbH (for 64 months)
• NUMLOG (for 64 months)
• WinGo AG (for 63 months)
• Ecole Centrale de Nantes LHEEA (for 59 months)
• Sig-I/O (for 57 months)
• Entr ouvert (for 54 months)
• Adfinis SyGroup AG (for 52 months)
• GNI MEDIA (for 46 months)
• Laboratoire LEGI UMR 5519 / CNRS (for 46 months)
• Tesorion (for 46 months)
• Bearstech (for 38 months)
• LiHAS (for 38 months)
• People Doc (for 34 months)
• Catalyst IT Ltd (for 32 months)
• Supagro (for 28 months)
• Demarcq SAS (for 26 months)
• TrapX Security (for 23 months)
• Universit Grenoble Alpes (for 12 months)
• TouchWeb SAS (for 4 months)
• SPiN AG

No comment Liked this article? Click here. My blog is Flattr-enabled.

" 'Boa constrictors swallow their prey whole, without chewing it. After that they are not able to move, and they sleep through the six months that they need for digestion.' I pondered deeply, then, over the adventures of the jungle. And after some work with a colored pencil I succeeded in making my first drawing. My Drawing Number One. It looked something like this: I showed my masterpiece to the grown-ups, and asked them whether the drawing frightened them. But they answered: 'Frighten? Why should any one be frightened by a hat?' My drawing was not a picture of a hat. It was a picture of a boa constrictor digesting an elephant. But since the grown-ups were not able to understand it, I made another drawing: I drew the inside of a boa constrictor, so that the grown-ups could see it clearly. They always need to have things explained. My Drawing Number Two looked like this: The grown-ups' response, this time, was to advise me to lay aside my drawings of boa constrictors, whether from the inside or the outside, and devote myself instead to geography, history, arithmetic, and grammar. That is why, at the age of six, I gave up what might have been a magnificent career as a painter. I had been disheartened by the failure of my Drawing Number One and my Drawing Number Two. Grown-ups never understand anything by themselves, and it is tiresome for children to be always and forever explaining things to them." from The Little Prince by Antoine de Saint Exup ry from the Corona Fun with Statistics department at Worldometers (source, archive.org link)

Welcome to the March 2020 report from the Reproducible Builds project. In our reports we outline the most important things that we have been up to over the past month and some plans for the future.

What are reproducible builds? One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes.

News The report from our recent summit in Marrakesh was published and is now available in both PDF and HTML formats. A sincere thank you to all of the Reproducible Builds community for the input to the event a sincere thank you to Aspiration for preparing and collating this report. Harmut Schorrig published a detailed document on how to compile Java applications in such as way that the .jar build artefact is reproducible across builds. A practical and hands-on guide, it details how to avoid unnecessary differences between builds by explicitly declaring an encoding as the default value differs across Linux and MS Windows systems and ensuring that the generated .jar a variant of a .zip archive does not embed any nondeterministic filesystem metadata, and so on. Janneke gave a quick presentation on GNU Mes and reproducible builds during the lighting talk session at LibrePlanet 2020. [ ] Vagrant Cascadian presented There and Back Again, Reproducibly! video at SCaLE 18x in Pasadena in California which generated some attention on Twitter. Herv Boutemy mentioned on our mailing list in a thread titled Rebuilding and checking Reproducible Builds from Maven Central repository that since the update of a central build script (the parent POM ) every Apache project using the Maven build system should build reproducibly. A follow-up discussion regarding how to perform such rebuilds was also started on the Apache mailing list. The Telegram instant-messaging platform announced that they had updated their iOS and Android OS applications and claim that they are reproducible according to their full instructions, verifying that its original source code is exactly the same code that is used to build the versions available on the Apple App Store and Google Play distribution platforms respectfully. Herv Boutemy also reported about a new project called reproducible-central which aims to allow anyone to rebuild a component from the Maven Central Repository that is expected to be reproducible and check that the result is as expected. In last month s report we detailed Omar Navarro Leija s work in and around an academic paper titled Reproducible Containers which describes in detail the workings of a user-space container tool called dettrace (PDF). Since then, the PhD student from the University Of Pennsylvania presented on this tool at the ASPLOS 2020 conference in Lausanne, Switzerland. Furthermore, there were contributions to dettrace from the Reproducible Builds community itself. [ ][ ]

Distribution work

openSUSE In openSUSE, Bernhard M. Wiedemann published his monthly Reproducible Builds status update as well as made the following changes within the distribution itself:

• avfs (report build problem in %post script)
• arj (fix incorrect use of strcpy, submitted upstream)
• brickv (update get upstream fix)
• fvwm-themes (delta between architectures in noarch package)
• libpeas (report build failure in single-CPU mode)
• pmix (update to incoporate upstream fix)
• pw3270 (date variation, forwarded upstream)
• python-mailmanclient (report build failure in single-CPU mode)
• ripgrep (CPU, forwarded upstream)
• tensorflow2 (avoid random temporary directory path)
• tesseract-ocr (drop native architecture optimisations)
• vlc (fixed ghost file size and sort archive, already upstream)

Debian Chris Lamb further refined his merge request for the debian-installer component to allow all arguments from sources.list files (such as [check-valid-until=no] ) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure. (#13) Holger Levsen filed a number of bug reports against the debrebuild tool that attempts to rebuild a Debian package given a .buildinfo file as input, including:

• Accepting signed .buildinfo files. (#955050)
• Two sbuild-related bugs. (#955123 & #955304)
• Specific adjustments to the APT configuration. (#955307, #955298 & #955280)
• Requests to improve the documentation in various ways. (#955049 & #955308)

48 reviews of Debian packages were added, 17 were updated and 34 were removed this month adding to our knowledge about identified issues. Many issue types were noticed, categorised and updated by Chris Lamb, including:

• nondeterministic_gtk_icon_cache [ ]
• nondeterministic_ordering_in_documentation_generated_by_doxygen [ ]
• nondeterministic_vo_files_generated_by_coq [ ] utput
• randomness_in_browserify_lite_output [ ]

Finally, Holger opened a bug report against the software running tracker.debian.org, a service for Debian Developers to follow the evolution of packages via web and email interfaces to request that they integrate information from buildinfos.debian.net (#955434) and Chris Lamb kept isdebianreproducibleyet.com up to date. [ ]

Software development

diffoscope Chris Lamb made the following changes to diffoscope, the Reproducible Builds project s in-depth and content-aware diff utility that can locate and diagnose reproducibility issues, including preparing and uploading version 138 to Debian:

• Improvements:
  • Don t allow errors with R script deserialisation cause the entire operation to fail, for example if an external library cannot be loaded. (#91)
  • Experiment with memoising output from expensive external commands, eg. readelf. (#93)
  • Use dumppdf from the python3-pdfminer if we do not see any other differences from pdftext, etc. (#92)
  • Prevent a traceback when comparing two R .rdx files directly as the get_member method will return a file even if the file is missing. [ ]
• Reporting:
  • Display the supported file formats into the package long description. (#90)
  • Print a potentially-helpful message if the PyPDF2 module is not installed. [ ]
  • Remove any duplicate comparator descriptions when formatting in the --help output or in the package long description. [ ]
  • Weaken Install the X package to get a better output message to may produce a better output as the former is not actually guaranteed. [ ]
• Misc:
  • Ensure we only parse the recommended packages from --list-debian-substvars when we want them for debian/tests/control generation. [ ]
  • Add upstream metadata file [ ] and add a Lintian override for upstream-metadata-in-native-source as we are upstream. [ ]
  • Inline the RequiredToolNotFound.get_package method s functionality as it is only used once. [ ]
  • Drop the deprecated py36 = [..] argument in the pyproject.toml file. [ ]

In addition, Vagrant Cascadian updated diffoscope in GNU Guix to version 138 [ ], as well as updating reprotest our end-user tool to build same source code twice in widely differing environments and then checks the binaries produced by each build for any differences to version 0.7.14 [ ].

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month we wrote a large number of such patches, including:

• Bernhard M. Wiedemann (via openSUSE):
  • arj (date variation)
  • gnulib (date variation)
  • gnulib (date variation)
  • lasso (sort filesystem ordering)
  • mono/at-spi-sharp (report nondeterminism from filesystem nondeterminism)
  • python-M2Crypto (report security certs expiring in 2029)
  • python-swifter (report single-CPU build failure)
  • QT uic (report ASLR nondeterminism)
  • tdiff (report single-CPU build failure)
  • tensorflow (report ASLR-induced variation)
  • volk (drop compile-time CPU detection)
• Chris Lamb:
  • #952990 filed against pmemkv (forwarded upstream)
  • #953071 filed against ndisc6.
  • #953117 filed against infernal.
  • #953263 filed against beep.
  • #953646 filed against node-nodedbi.
  • #954409 filed against node-browserify-lite.
  • #955009 filed against font-manager.
  • #955287 filed against pdb2pqr.
  • #955341 filed against gucharmap.
  • #955364 filed against cloudkitty.
  • isbg (report a non-determinstic documentation issue)

Project documentation There was further work performed on our documentation and website this month including Alex Wilson adding a section regarding using Gradle for reproducible builds in JVM projects [ ] and Holger Levsen added the report from our recent summit in Marrakesh [ ][ ]. In addition, Chris Lamb made a number of changes, including correcting the syntax of some CSS class formatting [ ], improved some filed against copy a little better [ ] and corrected a reference to calendar.monthrange Python method in a utility function. [ ]

Testing framework We operate a large and many-featured Jenkins-based testing framework that powers tests.reproducible-builds.org that, amongst many other tasks, tracks the status of our reproducibility efforts as well as identifies any regressions that have been introduced. This month, Chris Lamb reworked the web-based package rescheduling tool to:

• Require a HTTP POST method in the web-based scheduler as not only should HTTP GET requests be idempotent but this will allow many future improvements in the user interface. [ ][ ][ ]
• Improve the authentication error message in said rescheduler to suggest that the developer s SSL certificate may have expired. [ ]

In addition, Holger Levsen made the following changes:

• Add a new ath97 subtarget for the OpenWrt distribution.
• Revisit ordering of Debian suites; sort the experimental distribution last and reverse the ordering of suites to prioritise the suites in development. [ ][ ][ ]
• Schedule Debian buster and bullseye a little less in order to allow unstable to catch up on the i386 architecture. [ ][ ]
• Various cosmetic changes to the web-based scheduler. [ ][ ][ ][ ]
• Improve wordings in the node health maintenance output. [ ]

Lastly, Vagrant Cascadian updated a link to the (formerly) weekly news to our reports page [ ] and kpcyrd fixed the escaping in an Alpine Linux inline patch [ ]. The usual build nodes maintenance was performed by Holger Levsen [ ][ ], Mattia Rizzolo [ ] and Vagrant Cascadian [ ][ ].
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net.
• Twitter: @ReproBuilds
• Reddit: /r/ReproducibleBuilds
• Mailing list: rb-general@lists.reproducible-builds.org

This month s report was written by Bernhard M. Wiedemann, Chris Lamb, Holger Levsen and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

Over the past few weeks things have been pretty hectic. Since I'm not working at the moment I'm mostly doing childcare instead. I need a break, now and again, so I've been sending our child to p iv koti two days a week with him home the rest of the time. I love taking care of the child, because he's seriously awesome, but it's a hell of a lot of work when most of our usual escapes are unavailable. For example we can't go to the (awesome) Helsinki Central Library as that is closed. Instead of doing things outdoors we've been baking bread together, painting, listening to music and similar. He's a big fan of any music with drums and shouting, so we've been listening to Rammstein, The Prodigy, and as much Queen as I can slip in without him complaining ("more bang bang!"). I've also signed up for some courses at the Helsinki open university, including Devops with Docker so perhaps I have a future career working with computers? I'm hazy. Finally I saw a fun post the other day on reddit asking about the creation of a DSL for server-setup. I wrote a reply which basically said two things:

• First of all you need to define the minimum set of primitives you can execute.
  • (Creating a file, fetching a package, reloading services when a configuration file changes, etc.)
• Then you need to define a syntax for expressing those rules.
  • Not using YAML. Because Ansible fucked up bigtime with that.
  • It needs to be easy to explain, it needs to be consistent, and you need to decide before you begin if you want "toy syntax" or "programming syntax".
  • Because adding on conditionals, loops, and similar, will ruin everything if you add it once you've started with the wrong syntax. Again, see Ansible.

Anyway I had an idea of just expressing things in a simple fashion, borrowing Puppet syntax (which I guess is just Ruby hash literals). So a module to do stuff with files would just look like this:

file   name   => "This is my rule",
       target => "/tmp/blah",
       ensure => "absent"  

The next thing to do is to allow that to notify another rule, when it results in a change. So you add in:

notify => "Name of rule"
# or
notify => [ "Name of rule", "Name of another rule" ]

You could also express dependencies the other way round:

shell   name => "Do stuff",
        command => "wc -l /etc/passwd > /tmp/foo",
        requires => [ "Rule 1", "Rule 2"]  

Anyway the end result is a simple syntax which allows you to do things; I wrote a file to allow me to take a clean system and configure it to run a simple golang application in an hour or so. The downside? Well the obvious one is that there's no support for setting up cron jobs, setting up docker images, MySQL usernames/passwords, etc. Just a core set of primitives. Adding new things is easy, but also an endless job. So I added the ability to run external/binary plugins stored outside the project. To support that is simple with the syntax we have:

• We pass the parameters, as JSON, to STDIN of the binary.
• We read the result from STDOUT
  • Did the rule result in a change to the system?
  • Or was it a NOP?

All good. People can write modules, if they like, and they can do that in any language they like. Fun times. We'll call it marionette since it's all puppet-inspired:

• https://github.com/skx/marionette

And that concludes this irregular update.

Sometimes, in our day-to-day-interactions, communication becomes disruptive, resembling a chain of reactions to reactions to reactions. Sometimes we lose the capacity to express our ideas and feelings. Sometimes communication just gets stuck, maybe conflict breaks out. When we see these same patterns over and over again, this might be due to the ever same roles that we adopt and play. Learnt in childhood, these roles are deeply ingrained in our adult selves, and acted out as unconscious scripts. Until we notice and work on them. This is a post inspired by contents from my mediation training. In the 1960s, Stephen Karpman has thought of a model of human communication that maps the destructive interactions which occur between people. This map is known as the drama triangle. Karpman defined three roles that interact with each other. We can play one role at work, and a different one at home, and another one with our children. Or we can switch from one role to the other in just one conversation. The three roles are:

• The Persecutor. I'm right. It's all your fault. The Persecutor acts out criticism, accusation, and condemnation. Their behavior is controlling, blaming, shaming, oppressive, hurtful, angry, authoritarian, superior. They know everything better, they laugh about others, bully, shame, or belittle them. The Persecutor discounts others' value, looking down on them. Persecutor's thought: I'm okay, you're not okay.
• The Victim. I'm blameless. Poor me. The Victim feels not accepted by others, oppressed, helpless, hopeless, powerless, ashamed, inferior. The Victim thinks they are unable or not good enough to solve problems on their own. The Victim discounts themselves. Victim's thought: I'm not okay, you're okay.
• The Rescuer. I'm good. Let me help you! The Rescuer is a person who has unsolicited and unlimited advice concerning the Victim's problems. They think for the Victim, and comfort them, generally without having been asked to do so. The Rescuer acts seemingly to help the Victim but rescuing mostly helps them to feel better themselves, as it allows them to ignore their own anxieties, worries, or shortcomings. The Rescuer needs a Victim to rescue, effectively keeping the Victim powerless. The Rescuer discounts others' abilities to think and act for themselves, looking down on them. Rescuer's thought: I'm okay, you're not okay.

Does this sound familiar? "Involvement in an unhealthy drama triangle is not something another person is doing to you. It's something you are doing with another person or persons." Well, to be more precise, it's something that we are all doing to each other: "Drama triangles form when participants who are predispositioned to adopt the roles of a drama triangle come together over an issue." (quoted from: Escaping conflict and the Karpman Drama Triangle.) People act out these roles to meet personal (often unconscious) needs. But each of these roles is toxic in that it sees others as problems to react to. In not being able to see that we take on these roles, we keep the triangle going, like in a dispute in which one word provokes another until someone leaves, slamming the door. This is drama. When we are stuck in the drama triangle, no one wins because all three roles "cause pain", "perpetuate shame [and] guilt", and "keep people caught in dysfunctional behavior" (quoted from Lynne Namka: The Drama Triangle, Three Roles of Victim-hood). How to get out of the drama triangle Awareness. To get out of the triangle, it is foremost suggested to be aware of its existence. I agree, it helps. I see it everywhere now. Identifying one's role and starting to act differently. While we switch roles, we generally take on a preferred role that we act out most of the time, and that was learnt in childhood. (I found a test to identify one's common primary role in German.) But how do we act differently? We need to take another look at that uncanny triangle. From the drama triangle to the winner triangle I found it insightful to ask what benefit each role could potentially bring into the interaction.
Acey Choy has created the Winner triangle, in 1990, as an attempt to transform social interactions away from drama. Her winner triangle shifts our perceptions of the roles: the Victim becomes the Vulnerable, the Rescuer becomes the Caring, the Persecutor becomes the Assertive.

Persecutor            Rescuer       Assertive              Caring
I'm right.            I'm good.     I have needs.          I'm listening.
    ----------------------              ----------------------
    \                    /              \                    /
     \                  /                \                  /
      \                /                  \                /
       \              /                    \              /
        \            /                      \            /
         \          /                        \          /
          \        /                          \        /
           \      /                            \      /
            \    /                              \    /
             \  /                                \  /
              \/                                  \/
            Victim                            Vulnerable
            I'm blameless.                    I'm struggling.
Karpman Dreaded Drama Triangle            Choy's Winner Triangle

The Assertive "I have needs." has a calling, aims at change, initiates, and gives feedback. Skills to learn: The Assertive needs to learn to identify their needs, communicate them, and negotiate with others on eye level without shaming, punishing, or belittling them. The Assertive needs to learn to give constructive feedback, without dismissing others. (In the workplace, it could be helpful to have a space for this.) The Assertive could benefit from learning to use I-Statements. The Caring "I'm listening." shows good will and sensitivity, cares, is empathic and supportive. Skills to learn: The Caring needs to learn to respect the boundaries of others: trusting their abilities to think, problem solve and talk for themselves. Therefore, the Caring could benefit from improving their active listening skills. Furthermore the Caring needs to learn to identify and respect their own boundaries and not to do things only because it makes them feel better about themselves. The Vulnerable "I'm struggling." has the skill of seeing and naming problems. Skills to learn: The Vulnerable needs to learn to acknowledge their feelings and needs, practice self-awareness, and self-compassion. They need to untie their self-esteem from the validation of other people. They need to learn to take care of themselves, and to strengthen their problem solving and decision making skills. What has this got to do with autonomy and power structures? Each of these interactions is embedded in larger society, and, as said above, we learn these roles from childhood. Therefore, we perpetually reproduce power structures, and learnt behavior. I doubt that fixing this on an individual level is sufficient to transform our interactions outside of small groups, families or work places. Although that would be a good start. We can see that the triangle holds together because the Victim, seemingly devoid of a way to handle their own needs, transfers care of their needs to the Rescuer, thereby giving up on their autonomy. The Rescuer is provided by the Victim with a sense of autonomy, knowledge, and power, that only works while denying the Victim their autonomy. At the same time, the Persecutor denies everyone else's needs and autonomy, and feels powerful by dismissing others. I've recently mentioned the importance of autonomy in order to avoid burnout, and as a means to control one's own life. If the Rescuer can acknowledge being in the triangle, and give the Victim autonomy, by supporting them with compassion, empathy, and guidance, and at the same time respecting their own boundaries, we could find even more ways to escape the drama triangle. Notes My description of the roles was heavily inspired by the article Escaping Conflict and the Karpman Drama Triangle that has a lot more detail on how to escape the triangle, and how to recognize when we're moving into one of the roles. While the article is informing families living with a person suffering from a spectrum of Borderline Personality Disorder, the content applies to any dysfunctional interaction.