Search Results: "ema"

16 May 2020

Russ Allbery: DocKnot 3.04

This is a relatively small feature release of my tool for managing software documentation and releases. I'm slowly moving all of my packages from Travis-CI to GitHub Workflows for automated CI. GitHub Workflows is much easier to configure and control, and I've been a bit worried about the future of Travis-CI since their acquisition. It seems unlikely that GitHub Workflows is going anywhere. It would be nice to use a fully free software solution for CI, but there doesn't seem to be anything out there that's nearly as easy and straightforward to use, and I have neither the time nor the energy to cobble something together myself. The configuration is fairly straightforward and should be portable to any fully free solution that might materialize in the future. Anyway, as part of that migration I needed to make some changes to DocKnot to generate status badges from GitHub Workflows instead of Travis-CI. This release includes those changes. There is a backward-incompatible change to make the semantics of the package metadata a bit more straightforward: vcs.travis needs to be changed to vcs.status.travis. You can get the latest release from the DocKnot distribution page. Debian packages have been uploaded to my personal repository. I plan on uploading DocKnot to Debian proper once I change the metadata format to use YAML instead of relaxed JSON.

15 May 2020

Dirk Eddelbuettel: Let s celebrate Anna!

Today is graduation at Washington University, and just like many other places, the ceremonies are a lot more virtual and surreal that in other years. For Anna today marks the graduation from Biomedical Engineering with a BSc. The McKelvey School of Engineering put a Zoom meeting together yesterday which was nice, and there is something more virtual here. Hopefully a real-life commencenment can take place in a year the May 30, 2021, date has been set. The university also sent out a little commencement site/video which was cute. But at end of the day online-only still falls short of the real deal as we all know too well by now. During those years, just about the only thing really I ever tweeted about appears to be soccer related. As it should because ball is life, as we all know. Here is one from 1 1/2 years ago when her Club Team three-peated in their NIRSA division:

And that opens what may be the best venue for mocking Anna: this year, which her a senior and co-captain, the team actually managed to loose a league game (a shocking first in these years) and to drop the final. I presume they anticipated that all we would all talk about around now is The Last Dance and three-peats, and left it at that. Probably wise. Now just this week, and hence days before graduating with her B.Sc., also marks the first time Anna was addressed as Dr Eddelbuettel. A little prematurely I may say, but not too shabby to be in print already!

But on the topic of gratulations and what comes next, this tweet was very sweet:

As was this, which marked another impressive score:

So big thanks from all of us to WashU for being such a superb environment for Anna for those four years, and especially everybody at the Pappu Lab for giving Anna a home and base to start a research career. And deepest and most sincere congratulations to Anna before the next adventure starts .

13 May 2020

Mike Gabriel: Q: Remote Support Framework for the GNU/Linux Desktop?

TL;DR; For those (admins) of you who run GNU/Linux on staff computers: How do you organize your graphical remote support in your company? Get in touch, share your expertise and experiences. Researching on FLOSS based Linux Desktops When bringing GNU/Linux desktops to a generic folk of productive office users on a large scale, graphical remote support is a key feature when organizing helpdesk support teams' workflows. In a research project that I am currently involved in, we investigate the different available remote support technologies (VNC screen mirroring, ScreenCasts, etc.) and the available frameworks that allow one to provide a remote support infrastructure 100% on-premise. In this research project we intend to find FLOSS solutions for everything required for providing a large scale GNU/Linux desktop to end users, but we likely will have to recommend non-free solutions, if a FLOSS approach is not available for certain demands. Depending on the resulting costs, bringing forth a new software solution instead of dumping big money in subscription contracts for non-free software is seen as a possible alternative. As a member of the X2Go upstream team and maintainer of several remote desktop related tools and frameworks in Debian, I'd consider myself as sort of in-the-topic. The available (as FLOSS) underlying technologies for plumbing a remote support framework are pretty much clear (x11vnc, recent pipewire-related approaches in Wayland compositors, browser-based screencasting). However, I still lack a good spontaneous answer to the question: "How to efficiently software-side organize a helpdesk scenario for 10.000+ users regarding graphical remote support?". Framework for Remote Desktop in Webbrowsers In fact, in the context of my X2Go activities, I am currently planning to put together a Django-based framework for running X2Go sessions in a web browser. The framework that we will come up with (two developers have already been hired for an initial sprint in July 2020) will be designed to be highly pluggable and it will probably be easy to add remote support / screen sharing features further on. And still, I walk around with the question in mind: Do I miss anything? Is there anything already out there that provides a remote support solution as 100% FLOSS, that has enterprise grade, that up-scales well, that has a modern UI design, etc. Something that I simply haven't come across, yet? Looking forward to Your Feedback Please get in touch (OFTC/Freenode IRC, Telegram, Email), if you can fill the gap and feel like sharing your ideas and experiences. light+love
Mike

12 May 2020

Evgeni Golov: Building a Shelly 2.5 USB to TTL adapter cable

When you want to flash your Shelly 2.5 with anything but the original firmware for the first time, you'll need to attach it to your computer. Later flashes can happen over the air (at least with ESPHome or Tasmota), but the first one cannot. In theory, this is not a problem as the Shelly has a quite exposed and well documented interface: Shelly 2.5 pinout

However, on closer inspection you'll notice that your normal jumper wires don't fit as the Shelly has a connector with 1.27mm (0.05in) pitch and 1mm diameter holes. Now, there are various tutorials on the Internet how to build a compatible connector using Ethernet cables and hot glue or with female header socket legs, and you can even buy cables on Amazon for 18 ! But 18 sounded like a lot and the female header socket thing while working was pretty finicky to use, so I decided to build something different. We'll need 6 female-to-female jumper wires and a 1.27mm pitch male header. Jumper wires I had at home, the header I got is a SL 1X20G 1,27 from reichelt.de for 0.61 . It's a 20 pin one, so we can make 3 adapters out of it if needed. Oh and we'll need some isolation tape.

The first step is to cut the header into 6 pin chunks. Make sure not to cut too close to the 6th pin as the whole thing is rather fragile and you might lose it. SL 1X20G 1,27 cut into pieces

It now fits very well into the Shelly with the longer side of the pins. Shelly 2.5 with pin headers attached

Second step is to strip the plastic part of one side of the jumper wires. Those are designed to fit 2.54mm pitch headers and won't work for our use case otherwise. jumper wire with removed plastic

As the connectors are still too big, even after removing the plastic, the next step is to take some pliers and gently press the connectors until they fit the smaller pins of our header. Shelly 2.5 with pin headers and a jumper wire attached

Shelly 2.5 with pin headers and a jumper wire attached

Now is the time to put everything together. To avoid short circuiting the pins/connectors, apply some isolation tape while assembling, but not too much as the space is really limited. Shelly 2.5 with pin headers and a jumper wire attached and taped

Shelly 2.5 with pin headers and a jumper wire attached and taped

And we're done, a wonderful (lol) and working (yay) Shelly 2.5 cable that can be attached to any USB-TTL adapter, like the pictured FTDI clone you get almost everywhere. Shelly 2.5 with full cable and FTDI attached

Shelly 2.5 with full cable and FTDI attached

Yes, in an ideal world we would have soldered the header to the cable, but I didn't feel like soldering on that limited space. And yes, shrink-wrap might be a good thing too, but again, limited space and with isolation tape you only need one layer between two pins, not two.

Daniel Silverstone: The Lars, Mark, and Daniel Club

Last night, Lars, Mark, and I discussed Jeremy Kun's The communicative value of using Git well post. While a lot of our discussion was spawned by the article, we did go off-piste a little, and I hope that my notes below will enlighten you all as to a bit of how we see revision control these days. It was remarkably pleasant to read an article where the comments section wasn't a cesspool of horror, so if this posting encourages you to go and read the article, don't stop when you reach the bottom -- the comments are good and useful too.

This was a fairly non-contentious article for us though each of us had points we wished to bring up and chat about it turned into a very convivial chat. We saw the main thrust of the article as being about using the metadata of revision control to communicate intent, process, and decision making. We agreed that it must be possible to do so effectively with Mercurial (thus deciding that the mention of it was simply a bit of clickbait / red herring) and indeed Mark figured that he was doing at least some of this kind of thing way back with CVS. We all discussed how knowing the fundamentals of Git's data model improved our ability to work wih the tool. Lars and I mentioned how jarring it has originally been to come to Git from revision control systems such as Bazaar (bzr) but how over time we came to appreciate Git for what it is. For Mark this was less painful because he came to Git early enough that there was little more than the fundamental data model, without much of the porcelain which now exists. One point which we all, though Mark in particular, felt was worth considering was that of distinguishing between published and unpublished changes. The article touches on it a little, but one of the benefits of the workflow which Jeremy espouses is that of using the revision control system as an integral part of the review pipeline. This is perhaps done very well with Git based workflows, but can be done with other VCSs. With respect to the points Jeremy makes regarding making commits which are good for reviewing, we had a general agreement that things were good and sensible, to a point, but that some things were missed out on. For example, I raised that commit messages often need to be more thorough than one-liners, but Jeremy's examples (perhaps through expedience for the article?) were all pretty trite one-liners which perhaps would have been entirely obvious from the commit content. Jeremy makes the point that large changes are hard to review, and Lars poined out that Greg Wilson did research in this area, and at least one article mentions 200 lines as a maximum size of a reviewable segment. I had a brief warble at this point about how reviewing needs to be able to consider the whole of the intended change (i.e. a diff from base to tip) not just individual commits, which is also missing from Jeremy's article, but that such a review does not need to necessarily be thorough and detailed since the commit-by-commit review remains necessary. I use that high level diff as a way to get a feel for the full shape of the intended change, a look at the end-game if you will, before I read the story of how someone got to it. As an aside at this point, I talked about how Jeremy included a 'style fixes' commit in his example, but I loathe seeing such things and would much rather it was either not in the series because it's unrelated to it; or else the style fixes were folded into the commits they were related to. We discussed how style rules, as well as commit-bisectability, and other rules which may exist for a codebase, the adherence to which would form part of the checks that a code reviewer may perform, are there to be held to when they help the project, and to be broken when they are in the way of good communication between humans. In this, Lars talked about how revision control histories provide high level valuable communication between developers. Communication between humans is fraught with error and the rules are not always clear on what will work and what won't, since this depends on the communicators, the context, etc. However whatever communication rules are in place should be followed. We often say that it takes two people to communicate information, but when you're writing commit messages or arranging your commit history, the second party is often nebulous "other" and so the code reviewer fulfils that role to concretise it for the purpose of communication. At this point, I wondered a little about what value there might be (if any) in maintaining the metachanges (pull request info, mailing list discussions, etc) for historical context of decision making. Mark suggested that this is useful for design decisions etc but not for the style/correctness discussions which often form a large section of review feedback. Maybe some of the metachange tracking is done automatically by the review causing the augmentation of the changes (e.g. by comments, or inclusion of design documentation changes) to explain why changes are made. We discussed how the "rebase always vs. rebase never" feeling flip-flopped in us for many years until, as an example, what finally won Lars over was that he wants the history of the project to tell the story, in the git commits, of how the software has changed and evolved in an intentional manner. Lars said that he doesn't care about the meanderings, but rather about a clear story which can be followed and understood. I described this as the switch from "the revision history is about what I did to achieve the goal" to being more "the revision history is how I would hope someone else would have done this". Mark further refined that to "The revision history of a project tells the story of how the project, as a whole, chose to perform its sequence of evolution." We discussed how project history must necessarily then contain issue tracking, mailing list discussions, wikis, etc. There are exist free software projects where part of their history is forever lost because, for example, the project moved from Sourceforge to Github, but made no effort (or was unable) to migrate issues or somesuch. Linkages between changes and the issues they relate to can easily be broken, though at least with mailing lists you can often rewrite URLs if you have something consistent like a Message-Id. We talked about how cover notes, pull request messages, etc. can thus also be lost to some extent. Is this an argument to always use merges whose message bodies contain those details, rather than always fast-forwarding? Or is it a reason to encapsulate all those discussions into git objects which can be forever associated with the changes in the DAG? We then diverted into discussion of CI, testing every commit, and the benefits and limitations of automated testing vs. manual testing; though I think that's a little too off-piste for even this summary. We also talked about how commit message audiences include software perhaps, with the recent movement toward conventional commits and how, with respect to commit messages for machine readability, it can become very complex/tricky to craft good commit messages once there are multiple disparate audiences. For projects the size of the Linux kernel this kind of thing would be nearly impossible, but for smaller projects, perhaps there's value. Finally, we all agreed that we liked the quote at the end of the article, and so I'd like to close out by repeating it for you all... Hal Abelson famously said:

Programs must be written for people to read, and only incidentally for machines to execute.

Jeremy agrees, as do we, and extends that to the metacommit information as well.

Jacob Adams: Roman Finger Counting

I recently wrote a final paper on the history of written numerals. In the process, I discovered this fascinating tidbit that didn t really fit in my paper, but I wanted to put it somewhere. So I m writing about it here. If I were to ask you to count as high as you could on your fingers you d probably get up to 10 before running out of fingers. You can t count any higher than the number of fingers you have, right? The Romans could! They used a place-value system, combined with various gestures to count all the way up to 9,999 on two hands.

The System (Note that in this diagram 60 is, in fact, wrong, and this picture swaps the hundreds and the thousands.) We ll start with the units. The last three fingers of the left hand, middle, ring, and pinkie, are used to form them. Zero is formed with an open hand, the opposite of the finger counting we re used to. One is formed by bending the middle joint of the pinkie, two by including the ring finger and three by including the middle finger, all at the middle joint. You ll want to keep all these bends fairly loose, as otherwise these numbers can get quite uncomfortable. For four, you extend your pinkie again, for five, also raise your ring finger, and for six, you raise your middle finger as well, but then lower your ring finger. For seven you bend your pinkie at the bottom joint, for eight adding your ring finger, and for nine, including your middle finger. This mirrors what you did for one, two and three, but bending the finger at the bottom joint now instead. This leaves your thumb and index finger for the tens. For ten, touch the nail of your index finger to the inside of your top thumb joint. For twenty, put your thumb between your index and middle fingers. For thirty, touch the nails of your thumb and index fingers. For forty, bend your index finger slightly towards your palm and place your thumb between the middle and top knuckle of your index finger. For fifty, place your thumb against your palm. For sixty, leave your thumb where it is and wrap your index finger around it (the diagram above is wrong). For seventy, move your thumb so that the nail touches between the middle and top knuckle of your index finger. For eighty, flip your thumb so that the bottom of it now touches the spot between the middle and top knuckle of your index finger. For ninety, touch the nail of your index finger to your bottom thumb joint. The hundreds and thousands use the same positions on the right hand, with the units being the thousands and the tens being the hundreds. One account, from which the picture above comes, swaps these two, but the first account we have uses this ordering. Combining all these symbols, you can count all the way to 9,999 yourself on just two hands. Try it!

History

The Venerable Bede The first written record of this system comes from the Venerable Bede, an English Benedictine monk who died in 735. He wrote De computo vel loquela digitorum, On Calculating and Speaking with the Fingers, as the introduction to a larger work on chronology, De temporum ratione. (The primary calculation done by monks at the time was calculating the date of Easter, the first Sunday after the first full moon of spring). He also includes numbers from 10,000 to 1,000,000, but its unknown if these were inventions of the author and were likely rarely used regardless. They require moving your hands to various positions on your body, as illustrated below, from Jacob Leupold s Theatrum Arilhmetico-Geometricum, published in 1727:

The Romans If Bede was the first to write it, how do we know that it came from Roman times? It s referenced in many Roman writings, including this bit from the Roman satirist Juvenal who died in 130:
Felix nimirum qui tot per saecula mortem distulit atque suos iam dextera computat annos. Happy is he who so many times over the years has cheated death And now reckons his age on the right hand.
Because of course the right hand is where one counts hundreds! There s also this Roman riddle:
Nunc mihi iam credas fieri quod posse negatur: octo tenes manibus, si me monstrante magistro sublatis septem reliqui tibi sex remanebunt. Now you shall believe what you would deny could be done: In your hands you hold eight, as my teacher once taught; Take away seven, and six still remain.
If you form eight with this system and then remove the symbol for seven, you get the symbol for six!

Sources My source for this blog post is Paul Broneer s 1969 English translation of Karl Menninger s Zahlwort und Ziffer (Number Words and Number Symbols).

11 May 2020

Markus Koschany: My Free Software Activities in April 2020

Welcome to gambaru.de. Here is my monthly report (+ the first week in May) that covers what I have been doing for Debian. If you re interested in Java, Games and LTS topics, this might be interesting for you. Debian Games

Scott Talbert did a fantastic job by porting playonlinux, a user-friendly frontend for Wine, to Python 3 (#937302). I tested his patch and uploaded the package today. More testing and feedback is welcome. Scott s work basically prevented the removal of one of the most popular packages in the games section. I believe this will also give interested people more time to package the Java successor of playonlinux called Phoenicis.
Reiner Herrmann ported ardentryst, an action role playing game, to Python 3 to fix a release critical Py2 removal bug (#936148). He also packaged the latest release of xaos, a real-time interactive fractal zoomer, and improved various packaging details. I reviewed both of them and sponsored the upload for him.
I packaged new upstream releases of minetest, lordsawar, gtkatlantic and cutemaze.
I also sponsored a new simutrans update for J rg Frings-F rst.

Debian Java

New upstream versions this month: jboss-modules, robocode, sweethome3d, sweethome3d-furniture-editor, jboss-logmanager, jboss-threads and undertow.
I prepared the security updates for tomcat9 (DSA-4680-1) and apache-log4j1.2.

Misc

I packaged new versions of wabt and binaryen, required to build Webassembly code from source.

Debian LTS This was my 50. month as a paid contributor and I have been paid to work 11,5 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following:

I completed the security update of Tomcat 8 in Stretch released as DSA-4673-1 and Tomcat 8 in Jessie soon to be released as DLA-2209-1.
I am currently assigned more hours and my plan is to invest the time in a project to improve our knowledge about embedded code copies and their current security impact which I want to discuss with the security team. The rest will be spent on Stretch security updates which will become the new LTS release soon.

ELTS Extended Long Term Support (ELTS) is a project led by Freexian to further extend the lifetime of Debian releases. It is not an official Debian project but all Debian users benefit from it without cost. The current ELTS release is Debian 7 Wheezy . This was my 23. month and I have been paid to work 2 hours on ELTS.

I prepared the fix for CVE-2019-18218 in php5 released as ELA-227-1.
I checked jetty for unfixed vulnerabilities and discovered that the version in Wheezy was not affected by CVE-2019-17632. No further action was required.
It turned out that the apache2 package in Wheezy was not affected by vulnerable embedded expat code because it depends on already fixed system packages.

Thanks for reading and see you next time.

10 May 2020

Enrico Zini: Fraudsters and pirates

Adele Spitzeder - Wikipedia

history people archive.org

2020-05-11

Adelheid Luise "Adele" Spitzeder ([ a dl ha t a de l p t tse d ]; 9 February 1832 27 or 28 October 1895), also known by her stage name Adele Vio, was a German actress, folk singer, and con artist. Initially a promising young actress, Spitzeder became a well-known private banker in 19th-century Munich when her theatrical success dwindled. Running what was possibly the first recorded Ponzi scheme, she offered large returns on investments by continually using the money of new investors to pay back the previous ones. At the height of her success, contemporary sources considered her the wealthiest woman in Bavaria.

Anne Bonny - Wikipedia

history people archive.org

2020-05-11

Anne Bonny (possibly 1697 possibly April 1782)[1][2] was an Irish pirate operating in the Caribbean, and one of the most famous female pirates of all time.[3] The little that is known of her life comes largely from Captain Charles Johnson's A General History of the Pyrates.

Mary Read - Wikipedia

history people archive.org

2020-05-11

Mary Read (1685 28 April 1721), also known as Mark Read, was an English pirate. She and Anne Bonny are two of the most famed female pirates of all time, and among the few women known to have been convicted of piracy during the early 18th century, at the height of the "Golden Age of Piracy".

Women in piracy - Wikipedia

history people archive.org

2020-05-11

While piracy was predominantly a male occupation, a minority of pirates were women.[1] On many ships, women (as well as young boys) were prohibited by the ship's contract, which all crew members were required to sign.[2] :303

Russ Allbery: Review: Golden Gates

Review: Golden Gates, by Conor Dougherty

Publisher:	Penguin
Copyright:	2020
ISBN:	0-525-56022-X
Format:	Kindle
Pages:	249

This review, for reasons that will hopefully become clear later, starts with a personal digression. I have been interested in political theory my entire life. That sounds like something admirable, or at least neutral. It's not. "Interested" means that I have opinions that are generally stronger than my depth of knowledge warrants. "Interested" means that I like thinking about and casting judgment on how politics should be done without doing the work of politics myself. And "political theory" is different than politics in important ways, not the least of which is that political actions have rarely been a direct danger to me or my family. I have the luxury of arguing about politics as a theory. In short, I'm at high risk of being one of those people who has an opinion about everything and shares it on Twitter. I'm still in the process (to be honest, near the beginning of the process) of making something useful out of that interest. I've had some success when I become enough a part of a community that I can do some of the political work, understand the arguments at a level deeper than theory, and have to deal with the consequences of my own opinions. But those communities have been on-line and relatively low stakes. For the big political problems, the ones that involve governments and taxes and laws, those that decide who gets medical treatment and income support and who doesn't, to ever improve, more people like me need to learn enough about the practical details that we can do the real work of fixing them, rather than only making our native (and generally privileged) communities better for ourselves. I haven't found my path helping with that work yet. But I do have a concrete, challenging, local political question that makes me coldly furious: housing policy. Hence this book. Golden Gates is about housing policy in the notoriously underbuilt and therefore incredibly expensive San Francisco Bay Area, where I live. I wanted to deepen that emotional reaction to the failures of housing policy with facts and analysis. Golden Gates does provide some of that. But this also turns out to be a book about the translation of political theory into practice, about the messiness and conflict that results, and about the difficult process of measuring success. It's also a book about how substantial agreement on the basics of necessary political change can still founder on the shoals of prioritization, tribalism, and people who are interested in political theory. In short, it's a book about the difficulty of changing the world instead of arguing about how to change it. This is not a direct analysis of housing policy, although Dougherty provides the basics as background. Rather, it's the story of the political fight over housing told primarily through two lenses: Sonja Trauss, founder of BARF (the Bay Area Renters' Federation); and a Redwood City apartment complex, the people who fought its rent increases, and the nun who eventually purchased it. Around that framework, Dougherty writes about the Howard Jarvis Taxpayers Association and the history of California's Proposition 13, a fight over a development in Lafayette, the logistics challenge of constructing sufficient housing even when approved, and the political career of Scott Wiener, the hated opponent of every city fighting for the continued ability to arbitrarily veto any new housing. One of the things Golden Gates helped clarify for me is that there are three core interest groups that have to be part of any discussion of Bay Area housing: homeowners who want to limit or eliminate local change, renters who are vulnerable to gentrification and redevelopment, and the people who want to live in that area and can't (which includes people who want to move there, but more sympathetically includes all the people who work there but can't afford to live locally, such as teachers, day care workers, food service workers, and, well, just about anyone who doesn't work in tech). (As with any political classification, statements about collectives may not apply to individuals; there are numerous people who appear to fall into one group but who vote in alignment with another.) Dougherty makes it clear that housing policy is intractable in part because the policies that most clearly help one of those three groups hurt the other two. As advertised by the subtitle, Dougherty's focus is on the fight for more housing. Those who already own homes whose values have been inflated by artificial scarcity, or who want to preserve such stratified living conditions as low-density, large-lot single-family dwellings within short mass-transit commute of one of the densest cities in the United States, don't get a lot of sympathy or focus here except as opponents. I understand this choice; I also don't have much sympathy. But I do wish that Dougherty had spent more time discussing the unsustainable promise that California has implicitly made to homeowners: housing may be impossibly expensive, but if you can manage to reach that pinnacle of financial success, the ongoing value of your home is guaranteed. He does mention this in passing, but I don't think he puts enough emphasis on the impact that a single huge, illiquid investment that is heavily encouraged by government policy has on people's attitude towards anything that jeopardizes that investment. The bulk of this book focuses on the two factions trying to make housing cheaper: Sonja Trauss and others who are pushing for construction of more housing, and tenant groups trying to manage the price of existing housing for those who have to rent. The tragedy of Bay Area housing is that even the faintest connection of housing to the economic principle of supply and demand implies that the long-term goals of those two groups align. Building more housing will decrease the cost of housing, at least if you build enough of it over a long enough period of time. But in the short term, particularly given the amount of Bay Area land pre-emptively excluded from housing by environmental protection and the actions of the existing homeowners, building more housing usually means tearing down cheap lower-density housing and replacing it with expensive higher-density housing. And that destroys people's lives. I'll admit my natural sympathy is with Trauss on pure economic grounds. There simply aren't enough places to live in the Bay Area, and the number of people in the area will not decrease. To the marginal extent that growth even slows, that's another tale of misery involving "super commutes" of over 90 minutes each way. But the most affecting part of this book was the detailed look at what redevelopment looks like for the people who thought they had housing, and how it disrupts and destroys existing communities. It's impossible to read those stories and not be moved. But it's equally impossible to not be moved by the stories of people who live in their cars during the week, going home only on weekends because they have to live too far away from their jobs to commute. This is exactly the kind of politics that I lose when I take a superficial interest in political theory. Even when I feel confident in a guiding principle, the hard part of real-world politics is bringing real people with you in the implementation and mitigating the damage that any choice of implementation will cause. There are a lot of details, and those details matter. Without the right balance between addressing a long-term deficit and providing short-term protection and relief, an attempt to alleviate unsustainable long-term misery creates more short-term misery for those least able to afford it. And while I personally may have less sympathy for the relatively well-off who have clawed their way into their own mortgage, being cavalier with their goals and their financial needs is both poor ethics and poor politics. Mobilizing political opponents who have resources and vote locally isn't a winning strategy. Dougherty is a reporter, not a housing or public policy expert, so Golden Gates poses problems and tells stories rather than describes solutions. This book didn't lead me to a brilliant plan for fixing the Bay Area housing crunch, or hand me a roadmap for how to get effectively involved in local politics. What it did do is tell stories about what political approaches have worked, how they've worked, what change they've created, and the limitations of that change. Solving political problems is work. That work requires understanding people and balancing concerns, which in turn requires a lot of empathy, a lot of communication, and sometimes finding a way to make unlikely allies. I'm not sure how broad the appeal of this book will be outside of those who live in the region. Some aspects of the fight for housing generalize, but the Bay Area (and I suspect every region) has properties specific to it or to the state of California. It has also reached an extreme of housing shortage that is rivaled in the United States only by New York City, which changes the nature of the solutions. But if you want to seriously engage with Bay Area housing policy, knowing the background explained here is nearly mandatory. There are some flaws I wish Dougherty would have talked more about traffic and transit policy, although I realize that could be another book but this is an important story told well. If this somewhat narrow topic is within your interests, highly recommended. Rating: 8 out of 10

9 May 2020

Thorsten Alteholz: My Debian Activities in April 2020

FTP master This month I accepted 384 packages and rejected 47. The overall number of packages that got accepted was 457. Debian LTS This was my seventieth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian. This month my all in all workload has been 28.75h. During that time I did LTS uploads of:

[DLA 2183-1] libgsf security update for one CVE
[DLA 2184-1] jsch security update for one CVE
[DLA 2185-1] eog security update for one CVE
[DLA 2187-1] radicale security update for one CVE
[DLA 2186-1] ncmpc security update for one CVE
[DLA 2188-1] php5 security update for three CVEs
[DLA 2189-1] rzip security update for one CVE
[DLA 2195-1] w3m security update for two CVEs
[DLA 2194-1] yodl security update for one CVE
[DLA 2197-1] miniupnpc security update for one CVE
[DLA 2196-1] pound security update for one CVE

As there have been lots of no-dsa-CVEs I continued my work on wireshark. Last but not least I did some days of frontdesk duties. Debian ELTS This month was the twenty second ELTS month. During my small allocated time I only uploaded:

ELA-227-1 for php5 fixing four CVEs

I also did some days of frontdesk duties. Other stuff Unfortunately this month again strange things happened outside Debian and I only got some stuff done. I improved packaging of

I sponsored uploads of

ulfius

I uploaded the new package

puppet-module-cirrax-gitolite

On my Go challenge I uploaded:
golang-github-facebookgo-subset, golang-github-facebookgo-ensure, golang-github-shurcool-gopherjslib, golang-github-grafana-grafana-plugin-model, golang-github-crewjam-httperr, golang-github-hashicorp-terraform-svchost, golang-github-neelance-sourcemap, golang-github-neelance-astrewrite, golang-github-kisielk-gotool, golang-github-gopherjs-gopherjs, golang-github-yvasiyarov-newrelic-platform-go, golang-github-rhnvrm-simples3, golang-github-robfig-go-cache, golang-github-xorcare-pointer, golang-github-goburrow-serial

8 May 2020

Petter Reinholdtsen: Jami as a Zoom client, a trick for password protected rooms...

Half a year ago, I wrote about the Jami communication client, capable of peer-to-peer encrypted communication. It handle both messages, audio and video. It uses distributed hash tables instead of central infrastructure to connect its users to each other, which in my book is a plus. I mentioned briefly that it could also work as a SIP client, which came in handy when the higher educational sector in Norway started to promote Zoom as its video conferencing solution. I am reluctant to use the official Zoom client software, due to their copyright license clauses prohibiting users to reverse engineer (for example to check the security) and benchmark it, and thus prefer to connect to Zoom meetings with free software clients. Jami worked OK as a SIP client to Zoom as long as there was no password set on the room. The Jami daemon leak memory like crazy (approximately 1 GiB a minute) when I am connected to the video conference, so I had to restart the client every 7-10 minutes, which is not a great. I tried to get other SIP Linux clients to work without success, so I decided I would have to live with this wart until someone managed to fix the leak in the dring code base. But another problem showed up once the rooms were password protected. I could not get my dial tone signaling through from Jami to Zoom, and dial tone signaling is used to enter the password when connecting to Zoom. I tried a lot of different permutations with my Jami and Asterisk setup to try to figure out why the signaling did not get through, only to finally discover that the fundamental problem seem to be that Zoom is simply not able to receive dial tone signaling when connecting via SIP. There seem to be nothing wrong with the Jami and Asterisk end, it is simply broken in the Zoom end. I got help from a very skilled VoIP engineer figuring out this last part. And being a very skilled engineer, he was also able to locate a solution for me. Or to be exact, a workaround that solve my initial problem of connecting to password protected Zoom rooms using Jami. So, how do you do this, I am sure you are wondering by now. The trick is already documented from Zoom, and it is to modify the SIP address to include the room password. What is most surprising about this is that the automatically generated email from Zoom with instructions on how to connect via SIP do not mention this. The SIP address to use normally consist of the room ID (a number), an @ character and the IP address of the Zoom SIP gateway. But Zoom understand a lot more than just the room ID in front of the at sign. The format is "[Meeting ID].[Password].[Layout].[Host Key]", and you can hear see how you can both enter password, control the layout (full screen, active presence and gallery) and specify the host key to start the meeting. The full SIP address entered into Jami to provide the password will then look like this (all using made up numbers):

sip:657837644.522827@192.168.169.170

Now if only jami would reduce its memory usage, I could even recommend this setup to others. :) As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

6 May 2020

Reproducible Builds: Reproducible Builds in April 2020

Welcome to the April 2020 report from the Reproducible Builds project. In our regular reports we outline the most important things that we and the rest of the community have been up to over the past month. What are reproducible builds? One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. But whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into seemingly secure software during the various compilation and distribution processes.

News It was discovered that more than 725 malicious packages were downloaded thousands of times from RubyGems, the official channel for distributing code for the Ruby programming language. Attackers used a variation of typosquatting and replaced hyphens and underscores (for example, uploading a malevolent `atlas-client` in place of `atlas_client`) that executed a script that intercepted Bitcoin payments. (Ars Technica report) Bernhard M. Wiedemann launched `ismypackagereproducibleyet.org`, a service that takes a package name as input and displays whether the package is reproducible in a number of distributions. For example, it can quickly show the status of Perl as being reproducible on openSUSE but not in Debian. Bernhard also improved the documentation of his unreproducible package to add some example patches for hash issues. [ ]. There was a post on Chaos Computer Club s website listing Ten requirements for the evaluation of Contact Tracing apps in relation to the SARS-CoV-2 epidemic. In particular:
4. Transparency and verifiability: The complete source code for the app and infrastructure must be freely available without access restrictions to allow audits by all interested parties. Reproducible build techniques must be used to ensure that users can verify that the app they download has been built from the audited source code.
Elsewhere, Nicolas Boulenguez wrote a patch for the Ada programming language component of the GCC compiler to skip `-f.-prefix-map` options when writing Ada Library Information files. Amongst other properties, these `.ali` files embed the compiler flags used at the time of the build which results in the absolute build path being recorded via `-ffile-prefix-map`, `-fdebug-prefix-map`, etc. In the Arch Linux project, kpcyrd* reported that they held their first rebuilder workshop . The session was held on IRC and participants were provided a document with instructions on how to install and use Arch s `repro` tool. The meeting resulted in multiple people with no prior experience of Reproducible Builds validate their first package. Later in the month he also announced that it was now possible to run independent rebuilders under Arch in a hands-off, everything just works solution to distributed package verification. Mathias Lang submitted a pull request against `dmd`, the canonical compiler for the D programming languageto add support for our `SOURCE_DATE_EPOCH` environment variable as well the other C preprocessor tokens such `DATE`, `TIME` and `TIMESTAMP` which was subsequently merged. `SOURCE_DATE_EPOCH` defines a distribution-agnostic standard for build toolchains to consume and emit timestamps in situations where they are deemed to be necessary. [ ] The Telegram instant-messaging platform announced that they had updated to version 5.1.1 continuing their claim that they are reproducible according to their full instructions and therefore verifying that its original source code is exactly the same code that is used to build the versions available on the Apple App Store and Google Play distribution platforms respectfully. Lastly, Herv Boutemy reported that 97% of the current development versions of various Maven packages appear to have a reproducible build. [ ]

Distribution work In Debian this month, 89 reviews of Debian packages were added, 21 were updated and 33 were removed this month adding to our knowledge about identified issues. Many issue types were noticed, categorised and updated by Chris Lamb, including:

`captures_build_path_in_hd5_database_files`

`cargo_installs_crates2_json`

`nondeterministic_devhelp_documentation_generated_by_gtk_doc`

`ros_dynamic_reconfigure_captures_build_path`

In addition, Holger Levsen filed a feature request against `debrebuild`, a tool for rebuilding a Debian package given a `.buildinfo` file, proposing to add `--standalone` or `--one-shot-mode` functionality.
In openSUSE, Bernhard M. Wiedemann made the following changes:

`blender` (sort C `readdir` call, rejected upstream)

`guile/guix` (parallelism race condition)

`mingw32-filesystem/mingw32-binutils` (sort `readdir`, filesystem, toolchain)

`mingw64-filesystem/mingw64-binutils` (sort `readdir`, filesystem, toolchain)

`musescore` (non-deterministic `.zip` files)

`OBS` (FTBFS in rebuild)

`perl-Image-Sane` (report hung build on a single core VM)

`ruby2.7` (date, already upstream)

`vtk` (drop unreproducible `.pyc` file)

In Arch Linux, a rebuilder instance has been setup at reproducible.archlinux.org that is rebuilding Arch s `[core]` repository directly. The first rebuild has led to approximately 90% packages reproducible contrasting with 94% on the Reproducible Build s project own ArchLinux status page on `tests.reproducible-builds.org` that continiously builds packages and does not verify Arch Linux packages. More information may be found on the corresponding wiki page and the underlying decisions were explained on our mailing list.

Software development

diffoscope Chris Lamb made the following changes to diffoscope, the Reproducible Builds project s in-depth and content-aware diff utility that can locate and diagnose reproducibility issues (including preparing and uploading versions `139`, `140`, `141`, `142` and `143` to Debian which were subsequently uploaded to the backports repository):

Comparison improvements:

Dalvik `.dex` files can also serve as APK containers so restrict the narrower identification of `.dex` files to files ending with this extension and widen the identification of APK files to when file(1) discovers a Dalvik file. (#28)

Add support for Hierarchical Data Format (HD5) files. (#95)

Add support for `.p7c` and `.p7b` certificates. (#94)

Strip paths from the output of `zipinfo(1)` warnings. (#97)

Don t uselessly include the JSON similarity percentage if it is 0.0% . [ ]

Render multi-line difference comments in a way to show indentation. (#101)

Testsuite improvements:

Add `pdftotext` as a requirement to run the PDF `test_metadata` text. (#99)

apktool 2.5.0 changed the handling of output of XML schemas so update and restrict the corresponding test to match. (#96)

Explicitly list `python3-h5py` in `debian/tests/control.in` to ensure that we have this module installed during a test run to generate the fixtures in these tests. [ ]

Correct parsing of `./setup.py test --pytest-args` arguments. [ ]

Misc:

Capitalise Ordering differences only in text comparison comments. [ ]

Improve documentation of `FILE_TYPE_HEADER_PREFIX` and `FALLBACK_FILE_TYPE_HEADER_PREFIX` to highlight that only the first 16 bytes are used. [ ]

Michael Osipov created a well-researched merge request to return diffoscope to using `zipinfo` directly instead of piping input via `/dev/stdin` in order to ensure portability to the BSD operating system [ ]. In addition, Ben Hutchings documented how `--exclude` arguments are matched against filenames [ ] and Jelle van der Waa updated the LLVM test fixture difference for LLVM version 10 [ ] as well as adding a reference to the name of the `h5dump` tool in Arch Linux [ ]. Lastly, Mattia Rizzolo also fixed in incorrect build dependency [ ] and Vagrant Cascadian enabled diffoscope to locate the `openssl` and `h5dump` packages on GNU Guix [ ][ ], and updated diffoscope in GNU Guix to version 141 [ ] and 143 [ ].

strip-nondeterminism strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. In April, Chris Lamb made the following changes:

Add deprecation plans to all handlers documenting how or if they could be disabled and eventually removed, etc. (#3)

Normalise `*.sym` files as Java archives. (#15)

Add support for custom `.zip` filename filtering and exclude two patterns of files generated by Maven projects in fork mode. (#13)

disorderfs disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues. This month, Chris Lamb fixed a long-standing issue by not drop UNIX groups in FUSE multi-user mode when we are not root (#1) and uploaded version `0.5.9-1` to Debian unstable. Vagrant Cascadian subsequently refreshed disorderfs in GNU Guix to version 0.5.9 [ ].

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

Bernhard M. Wiedemann:

`elixir` (parallelism)

`gnutls` (build failure)

`moonjit/bcc` (compile-time CPU-detection)

`openstack` (backport of patch to drop unreproducible sphinx `.pickle` files)

`x3270` (merged, update date patch)

Chris Lamb:

#958301 filed against `dh-cargo`.

#956549 filed against `gmap`.

#956591 filed against `gpick`.

#956477 filed against `herbstluftwm`.

#956304 filed against `libcamera`.

#956589 filed against `libctl`.

#956408 filed against `minetest-mod-xdecor`.

#955783 filed against `netgen-lvs`.

#958110 filed against `nickle`.

#958381 filed against `nmrpflash`.

#958382 filed against `node-mqtt`.

#956473 filed against `sprai`.

#955501 filed against `yaz`.

#956583 filed against `xxhash`.

In addition, Bernhard informed the following projects that their packages are not reproducible:

`acoular` (report unknown non-determinism)

`cri-o` (report a date issue)

`gnutls` (report `certtool` being unable to extend certificates beyond 2049)

`gnutls` (report copyright year variation)

`libxslt` (report a bug about non-deterministic output from data corruption)

`python-astropy` (report a future build failure in 2021)

Project documentation This month, Chris Lamb made a large number of changes to our website and documentation in the following categories:

Community engagement improvements:

Update instructions to register for Salsa on our Contribute page now that the signup process has been overhauled. [ ]

Make it clearer that joining the `rb-general` mailing list is probably a first step for contributors to take. [ ]

Make our full contact information easier to find in the footer (#19) and improve text layout using bullets to separate sections [ ].

Accessibility:

To improve accessibility, make all links underlined. (#12)

Use an enhanced foreground/background contrast ratio of 7.04:1. (#11)

General improvements:

Add a new Academic publications page. (#22)

Add Trezor to our list of affiliated projects. (#26)

Add the JVM page to the documentation index (#17) and tidy the page itself a little [ ].

Add a GNU Libtool pointer to the Archive metadata documentation page. [ ]

Internals:

Move to using `jekyll-redirect-from` over manual redirect pages [ ][ ] and add a redirect from `/docs/buildinfo/` to `/docs/recording/`. (#23)

Limit the website self-check to not scan generated files [ ] and remove the old layout checker now that I have migrated all them [ ].

Move the news archive under the `/news/` namespace [ ] and improve formatting of archived news links [ ].

Various improvements to the draft template generation. [ ][ ][ ][ ]

In addition, Holger Levsen clarified exactly which month we ceased to do weekly reports [ ] and Mattia Rizzolo adjusted the title style of an event page [ ]. Marcus Hoffman also started a discussion on our website s issue tracker asking for clarification on embedded signatures and Chris Lamb subsequently replied and asked Marcus to go ahead and propose a concrete change.

Testing framework We operate a large and many-featured Jenkins-based testing framework that powers `tests.reproducible-builds.org` that, amongst many other tasks, tracks the status of our reproducibility efforts as well as identifies any regressions that have been introduced.

Chris Lamb:

Print the build environment prior to executing a build. [ ]

Drop a misleading `disorderfs-debug` prefix in log output when we change non-disorderfs things in the file and, as it happens, do not run disorderfs at all. [ ]

The CSS for the package report pages added a margin to all `<a>` HTML elements under `<li>` ones, which was causing a comma/bullet spacing issue. [ ]

Tidy the copy in the project links sidebar. [ ]

Holger Levsen:

General:

Install `jekyll-redirect-from` as it now needed by the reproducible-builds.org website. [ ]

Improve/correct log parsing rules. [ ][ ]

Debian:

Reduce scheduling frequency of the buster distribution on the `arm64` architecture, etc.. [ ][ ]

Show builds per day on a per-architecture basis for the last year on the Debian dashboard. [ ]

Drop the Subgraph OS package set as development halted in 2017 or 2018. [ ]

Update `debrebuild` to version from the latest version of `devscripts`. [ ][ ]

Add or improve various parts of the documentation. [ ][ ][ ]

Work on a Debian rebuilder:

Integrate `sbuild`. [ ][ ][ ][ ][ ]

Select a random `.buildinfo` file and attempt to build and compare the result. [ ][ ][ ][ ]

Improve output and related output formatting. [ ][ ][ ][ ][ ]

Outline next steps for the development of the tool. [ ][ ][ ]

Various refactoring and code improvements. [ ][ ][ ]

Lastly, Mattia Rizzolo fixed some log parsing code regarding potentially-harmless warnings from package installation [ ][ ] and the usual build node maintenance was performed by Holger Levsen [ ][ ][ ] and Mattia Rizzolo [ ][ ][ ].

Misc news On our mailing list this month, Santiago Torres asked whether we were still publishing releases of our tools to our website and Chris Lamb replied that this was not the case and fixed the issue. Later in the month Santiago also reported that the signature for the `disorderfs` package did not pass its GPG verification which was also fixed by Chris Lamb. Hans-Christoph Steiner of the Guardian Project asked whether there would be interest in making our website translatable which resulted in a WIP merge request being filed against the website and a discussion on how to track translation updates.
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

IRC: `#reproducible-builds` on `irc.oftc.net`.

Twitter: @ReproBuilds @reproducible_builds@fosstodon.org

Reddit: /r/ReproducibleBuilds

Mailing list: `rb-general@lists.reproducible-builds.org`

This month s report was written by Bernhard M. Wiedemann, Chris Lamb, Daniel Shahaf, Holger Levsen, Jelle van der Waa, kpcyrd, Mattia Rizzolo and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

Russell Coker: About Reopening Businesses

Currently there is political debate about when businesses should be reopened after the Covid19 quarantine. Small Businesses One argument for reopening things is for the benefit of small businesses. The first thing to note is that the protests in the US say I need a haircut not I need to cut people s hair . Small businesses won t benefit from reopening sooner. For every business there is a certain minimum number of customers needed to be profitable. There are many comments from small business owners that want it to remain shutdown. When the government has declared a shutdown and paused rent payments and provided social security to employees who aren t working the small business can avoid bankruptcy. If they suddenly have to pay salaries or make redundancy payouts and have to pay rent while they can t make a profit due to customers staying home they will go bankrupt. Many restaurants and cafes make little or no profit at most times of the week (I used to be 1/3 owner of an Internet cafe and know this well). For such a company to be viable you have to be open most of the time so customers can expect you to be open. Generally you don t keep a cafe open at 3PM to make money at 3PM, you keep it open so people can rely on there being a cafe open there, someone who buys a can of soda at 3PM one day might come back for lunch at 1:30PM the next day because they know you are open. A large portion of the opening hours of a most retail companies can be considered as either advertising for trade at the profitable hours or as loss making times that you can t close because you can t send an employee home for an hour. If you have seating for 28 people (as my cafe did) then for about half the opening hours you will probably have 2 or fewer customers in there at any time, for about a quarter the opening hours you probably won t cover the salary of the one person on duty. The weekend is when you make the real money, especially Friday and Saturday nights when you sometimes get all the seats full and people coming in for takeaway coffee and snacks. On Friday and Saturday nights the 60 seat restaurant next door to my cafe used to tell customers that my cafe made better coffee. It wasn t economical for them to have a table full for an hour while they sell a few cups of coffee, they wanted customers to leave after dessert and free the table for someone who wants a meal with wine (alcohol is the real profit for many restaurants). The plans of reopening with social distancing means that a 28 seat cafe can only have 14 chairs or less (some plans have 25% capacity which would mean 7 people maximum). That means decreasing the revenue of the most profitable times by 50% to 75% while also not decreasing the operating costs much. A small cafe has 2-3 staff when it s crowded so there s no possibility of reducing staff by 75% when reducing the revenue by 75%. My Internet cafe would have closed immediately if forced to operate in the proposed social distancing model. It would have been 1/4 of the trade and about 1/8 of the profit at the most profitable times, even if enough customers are prepared to visit and social distancing would kill the atmosphere. Most small businesses are barely profitable anyway, most small businesses don t last 4 years in normal economic circumstances. This reopen movement is about cutting unemployment benefits not about helping small business owners. Destroying small businesses is also good for big corporations, kill the small cafes and restaurants and McDonald s and Starbucks will win. I think this is part of the motivation behind the astroturf campaign for reopening businesses. Forbes has an article about this [1]. Psychological Issues Some people claim that we should reopen businesses to help people who have psychological problems from isolation, to help victims of domestic violence who are trapped at home, to stop older people being unemployed for the rest of their lives, etc. Here is one article with advice for policy makers from domestic violence experts [2]. One thing it mentions is that the primary US federal government program to deal with family violence had a budget of $130M in 2013. The main thing that should be done about family violence is to make it a priority at all times (not just when it can be a reason for avoiding other issues) and allocate some serious budget to it. An agency that deals with problems that affect families and only has a budget of $1 per family per year isn t going to be able to do much. There are ongoing issues of people stuck at home for various reasons. We could work on better public transport to help people who can t drive. We could work on better healthcare to help some of the people who can t leave home due to health problems. We could have more budget for carers to help people who can t leave home without assistance. Wanting to reopen restaurants because some people feel isolated is ignoring the fact that social isolation is a long term ongoing issue for many people, and that many of the people who are affected can t even afford to eat at a restaurant! Employment discrimination against people in the 50+ age range is an ongoing thing, many people in that age range know that if they lose their job and can t immediately find another they will be unemployed for the rest of their lives. Reopening small businesses won t help that, businesses running at low capacity will have to lay people off and it will probably be the older people. Also the unemployment system doesn t deal well with part time work. The Australian system (which I think is similar to most systems in this regard) reduces the unemployment benefits by $0.50 for every dollar that is earned in part time work, that effectively puts people who are doing part time work because they can t get a full-time job in the highest tax bracket! If someone is going to pay for transport to get to work, work a few hours, then get half the money they earned deducted from unemployment benefits it hardly makes it worthwhile to work. While the exact health impacts of Covid19 aren t well known at this stage it seems very clear that older people are disproportionately affected, so forcing older people to go back to work before there is a vaccine isn t going to help them. When it comes to these discussions I think we should be very suspicious of people who raise issues they haven t previously shown interest in. If the discussion of reopening businesses seems to be someone s first interest in the issues of mental health, social security, etc then they probably aren t that concerned about such issues. I believe that we should have a Universal Basic Income [3]. I believe that we need to provide better mental health care and challenge the gender ideas that hurt men and cause men to hurt women [4]. I believe that we have significant ongoing problems with inequality not small short term issues [5]. I don t think that any of these issues require specific changes to our approach to preventing the transmission of disease. I also think that we can address multiple issues at the same time, so it is possible for the government to devote more resources to addressing unemployment, family violence, etc while also dealing with a pandemic.

4 May 2020

Jonathan Dowland: Amiga floppy recovery project: what next?

This is the ninth part in a series of blog posts. The previous post was Amiga floppy recovery project scope. The whole series is available here: Amiga.

It's been a while since I've reported on my Amiga floppy recovery project. With my bulky Philips CRT attached I slowly ground through the process of importing all my floppy disks, which is now done. The majority of disks were imported without errors. Commercial disks were the most likely to fail to import. Possibly I'd have more success with them if I used a different copying technique than X-COPY's default, but my focus was not on the commercial disks. I have not yet restored the use of my LCD TV with the Amiga. I was waiting to hear back from Amiga Kit about an agreed return, but despite their web store claiming they're still open for business, I haven't been able to get any response to my emails to them since mid-February. I've given up, written off that order and bought an RGB/SCART adaptor elsewhere instead. Meanwhile my bulky CRT has returned to the loft. The next step is pretty boring: time to check over my spreadsheet of disks, the imported copies of them, make sure everything is accounted for and decide whether to make further attempts for any that failed or write them off. For commercial disks it's almost certainly less effort to track down someone else's rip than to try mine again. After that I can return to moving the Gotek floppy adaptor inside the Amiga case, as described in the last entry, and start playing.

Dima Kogan: Redirection and /dev/stderr

I just hit some unexpected unix-ism (or maybe Linux-ism?) that I'd like to mention here. I found a workaround, but if anybody knows what's actually happening and can enlighten me, please send me an email, and I'll update the post. OK, so I have a program that does stuff, and outputs its progress to stderr. And I'm invoking this from a shell script that also outputs stuff to stderr. A minimized sample:

echo "Outer print 1111" > /dev/stderr
perl -E 'say STDERR ("1111 1111 1111 1111 1111 1111 1111 1111\n")'
echo "Outer print 2222" > /dev/stderr
perl -E 'say STDERR ("2222 2222 2222 2222 2222 2222 2222 2222\n")'

Let's call this dothing.sh. This works just fine. Now let's say I want to log the output to a file. Naturally I do this:

./dothing.sh 2> /tmp/log

Surprisingly, this does not work:

$ bash dothing.sh 2> /tmp/log && cat /tmp/log
Outer print 2222
2222 2222 2222 2222 2222 2222 2222 2222
$ bash dothing.sh 2> /tmp/log && xxd /tmp/log
0000000: 4f75 7465 7220 7072 696e 7420 3232 3232  Outer print 2222
0000010: 0a00 0000 0000 0000 0000 0000 0000 0000  ................
0000020: 0000 0000 0000 0000 0032 3232 3220 3232  .........2222 22
0000030: 3232 2032 3232 3220 3232 3232 2032 3232  22 2222 2222 222
0000040: 3220 3232 3232 2032 3232 3220 3232 3232  2 2222 2222 2222
0000050: 0a0a                                     ..

The 1111 prints are gone, and we now have a block of binary 0 bytes in there for some reason. I had been assuming that writes to stderr block until they're finished. So I can, without thinking about it, mix writing to fd 2 and opening, and then writing to /dev/stderr. Apparently the right way to do this is to write to fd 2 in the script directly, instead of opening /dev/stderr:

echo "Outer print 1111" >&2
perl -E 'say STDERR ("1111 1111 1111 1111 1111 1111 1111 1111\n")'
echo "Outer print 2222" >&2
perl -E 'say STDERR ("2222 2222 2222 2222 2222 2222 2222 2222\n")'

Why?

Update So I poked at it for a bit more, and it turns out that the obvious scapegoat (buffering) is actually not the issue. The problem is this:

$ ls -l /dev/stderr(:A)
-rw-r--r-- 1 dima dima 36 May  4 17:02 /tmp/log

:A is the zsh syntax for recursive link-following. /dev/stderr is thus ultimately a link to /tmp/log. So the file on disk /tmp/log is being opened for writing and written-to twice at the same time

once in the outer 2>/tmp/log redirection
again in the inner >/dev/stderr redirection

Now we know. Thanks to Larry Doolittle for pointers.

3 May 2020

Enrico Zini: Tech links

Debops (2014) Hacker News

debian devel archive.org

2020-05-04

The way developers somehow think DevOps is (or should be) an abbreviation of "Developers doing/replacing Operations" is terrifying to me.I'm also in the same boat as the author, in that I recommend and target Debian Stable + Backports (and some vendor/community repos when required).

What nobody tells you about documentation - Blog - Divio

devel archive.org

2020-05-04

However hard you work on documentation, it won't work for your software - unless you do it the right way.

Eagle's Path: (2019-11-09)

devel python archive.org

2020-05-04

I'm going to preach the wonders of Python dataclasses, but for reasons of interested to those who have already gone down the delightful rabbit-hole of typed Python. So let me start with a quick plug for mypy if you haven't heard about it.

Encoding your WiFi access point password into a QR code

howto security archive.org

2020-05-04

Subscribe in a reader Subscribe by Email

My Abandonware: because old video games were better

games history archive.org

2020-05-04

Database of 15500 abandonware games free. One of the most complete video games museum. Take a trip down Memory Lane now! Warning: whole weekends can be lost.

Dirk Eddelbuettel: #0: Introducing T^4: Tips, Tricks, Tools, and Toys

For way too long now something I had meant to start was a little series about tips, tricks, tools, and toys. I had mentioned the idea a few times to a friend or two, and generally received a thumbs up or a go for it . But it takes a little to get over the humb and get going. And it turns out that last week s r^4 talk on upgrading to R 4.0.0 hit some latent demand as we are now at 1400 views on YouTube. Wowser. So hence without further ado, let s kick off T^4. Similar in spirit to R^4, but broader in scope and going beyond R. The opening slides explaining what we plan to do are here, and the video link follows below:

With some luck we should have the first actual talk next week. See you then! If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Sean Whitton: Processing mail from discussion lists differently

I realised this week that for some years I have been applying Inbox Zero indiscriminately to all e-mail that I receive, but this does not make sense, and has some downsides. My version of Inbox Zero works very well when applied to mail addressed directly to me, and for mail to certain mailing lists, where each post to the list might as well be addressed directly to me, in addition to the list. However, I also receive by e-mail the following things to which I now believe Inbox Zero should not be applied:

discussion lists like debian-devel, notmuch@notmuchmail.org, etc.
mail to aliases like ftpmaster@debian.org (except when that mail is in reply to mail written by me from that address)
automated notifications received via Debian team mailing lists, where I m not solely responsible for the Debian package in question, such as notifications received via the Debian Perl Group s mailing list
RSS feed articles supplied by my (years old but still going strong) rss2email cronjob.

I believe that applying Inbox Zero to these sorts of things is not only incorrect but is actually harming my engagement with these mediums. Let us distinguish

processing e-mail this means applying the Inbox Zero decision procedure to incoming messages, at set times during the day (I do it once, around 4pm)
browsing and sometimes catching up RSS articles and list mail looking through unread items, replying if I think I have something useful to say, leaving things for later, and occasionally marking all as read if I ve not had time for that group of lists lately.

These should be kept apart. The easy case to see is why you shouldn t apply the browsing/catching up approach to e-mail which should rather be processed that s just the original wisdom of Inbox Zero. And clearly you don t want ever to have to resort to just marking all mail addressed directly to you as read. What goes wrong, then, if you misapply the processing mentality to e-mail which should, rather, be browsed/caught up? Well, the core of Inbox Zero is deciding whether to read and reply to something right now, add it to your todo list to handle later, or decide the mail cannot be dealt with quickly but is not important enough to go on that todo list. If you apply this to RSS articles and discussion list mail, then the third option is basically ruled out, because almost nothing on mailing lists or on blogs that I follow is important enough to go on my list of Real Tasks. But then you re faced with either reading the article/post right now, or discarding it. There is no option to leave the item marked as unread and then maybe come back to it, or postpone discarding it until you ve decided to catch up the group. Applying Inbox Zero to discussion lists and RSS feeds creates a false sense of urgency. I ve realised that my implicit response to this has been reluctance to subscribe to new mailing lists and feeds, because I don t have things set up to allow me to read them in a leisurely way. But then I m missing out on discussions and writing that might be relevant and beneficial to me if I could only approach them when I m in a frame of mind other than time to get my inbox down to zero . This also means that subscribing to new mailing lists just in order to post something has an unreasonably high cost: introducing all mail on those lists into my Inbox Zero processing window. So, what s needed is to make the virtual folder views which I use to read new mail correspond cleanly to the distinction between mail to be processed and mail to be browsed. I.e. for each virtual folder it should be clear whether mail there is to be processed or to be browsed. Then I can continue to use my processing views once per day, and access the browsing views at leisure. (Indeed, I ve created a keybinding to cycle through the new browsing views, and another to catch them up.) As I mentioned, some list mail ought to be processed. For example, I want to process rather than browse the debian-policy mailing list, as one of the Policy Editors. With notmuch, it s easy to include this mail in the relevant processing views (I ve long had one processing view for weekdays and another for weekends). Additionally, I ve used a bit of Emacs Lisp to create a dynamic uncategorised unread view which catches mail to be browsed which (i) doesn t show up in one of the other virtual folders for mail to be browsed, and (ii) doesn t show up in one of the processing views. So now subscribing to a mailing list is cheap: mail will end up in the uncategorised view, and I can decide whether to leave it there for a low traffic list; create a new virtual folder for the list (trivial as it s all just more Emacs Lisp, no actual moving of files required); or unsubscribe. I ve been experiencing nostalgia for my time in secondary school when my friends and I had all sorts of interesting stuff flowing into our mailstores, from RSS feeds to each other s blogs where we d post both our own content and links elsewhere, RSS feeds to stranger s blogs, and e-mails sent to each other with links. We didn t have to worry about processing anything back then, as our e-mail accounts were used for intellectual enrichment but not the completion of tasks. I think I can recapture something of that with my new virtual folders for browsing.

Evgeni Golov: Remote management for OpenWRT devices without opening inbound connections

Everyone is working from home these days and needs a decent Internet connection. That's especially true if you need to do video calls and the room you want to do them has the worst WiFi coverage of the whole flat. Well, that's exactly what happened to my parents in law. When they moved in, we knew that at some point we'll have to fix the WiFi - the ISP provided DSL/router/WiFi combo would not cut it, especially not with the shape of the flat and the elevator shaft in the middle of it: the flat is essentially a big C around said shaft. But it was good enough for email, so we postponed that. Until now. The flat has wired Ethernet, but the users MacBook Air does not. That would have been too easy, right? So let's add another access point and hope the situation improves. Luckily I still had a TP-Link Archer C7 AC1750 in a drawer, which I could quickly flash with a fresh OpenWRT release, disable DHCPd and configure the same SSID and keys as the main/old WiFi. But I didn't know which channels would be best in the destination environment. Under normal circumstances, I'd just take the AP, drive to my parents in law and finish the configuration there. Nope, not gonna happen these days. So my plan was to finish configuration here, put the AP in a box and on the porch where someone can pick it up. But this would leave me without a way to further configure the device once it has been deployed - I was not particularly interested in trying to get port forwarding configured via phone and I was pretty sure UPnP was disabled in the ISP router. Installing a Tor hidden service for SSH was one possibility, setting up a VPN and making the AP a client another. Well, or just creating a reverse tunnel with SSH! sshtunnel Creating a tunnel with OpenSSH is easy: ssh -R127.0.0.1:2222:127.0.0.1:22 server.example.com will forward localhost:2222 on server.example.com to port 22 of the machine the SSH connection originated from. But what happens if the connection dies? Adding a while true; do ; done around it might help, but I would really like not to reinvent the wheel here! Thankfully, somebody already invented that particular wheel and OpenWRT comes with a sshtunnel package that takes care of setting up and keeping up such tunnels and documentation how to do so. Just install the sshtunnel package, edit /etc/config/sshtunnel to contain a server stanza with hostname, port and username and a tunnelR stanza referring said server plus the local and remote sides of the tunnel and you're good to go.

config server home
  option user     user
  option hostname server.example.com
  option port     22
config tunnelR local_ssh
  option server         home
  option remoteaddress  127.0.0.1
  option remoteport     2222
  option localaddress   127.0.0.1
  option localport      22

The only caveat is that sshtunnel needs the OpenSSH client binary (and the package correctly depends on it) and OpenWRT does not ship the ssh-keygen tool from OpenSSH but only the equivalent for Dropbear. As OpenSSH can't read Dropbear keys (and vice versa) you'll have to generate the key somewhere else and deploy it to the OpenWRT box and the target system. Oh, and OpenWRT defaults to enabling password login via SSH, so please disable that if you expose the box to the Internet in any way! Using the tunnel After configuring and starting the service, you'll see the OpenWRT system logging in to the configured remote and opening the tunnel. For some reason that connection would not show up in the output of w -- probably because there was no shell started or something, but logs show it clearly. Now it's just a matter of connecting to the newly open port and you're in. As the port is bound to 127.0.0.1, the connection is only possible from server.example.com or using it as a jump host via OpenSSH's ProxyJump option: ssh -J server.example.com -p 2222 root@localhost. Additionally, you can forward a local port over the tunneled connection to create a tunnel for the OpenWRT webinterface: ssh -J server.example.com -p 2222 -L8080:localhost:80 root@localhost. Yes, that's a tunnel inside a tunnel, and all the network engineers will go brrr, but it works and you can access LuCi on http://localhost:8080 just fine. If you don't want to type that every time, create an entry in your .ssh/config:

Host openwrt
  ProxyJump server.example.com
  HostName localhost
  Port 2222
  User root
  LocalForward 8080 localhost:80

And we're done. Enjoy easy access to the newly deployed box and carry on.

Fran ois Marier: Backing up to a GnuBee PC 2

After installing Debian buster on my GnuBee, I set it up for receiving backups from my other computers.

Software setup I started by configuring it like a typical server but without a few packages that either take a lot of memory or CPU:

I changed the default hostname:

/etc/hostname: foobar
/etc/mailname: foobar.example.com
/etc/hosts: 127.0.0.1 foobar.example.com vogar localhost

and then installed the avahi-daemon package to be able to reach this box using foobar.local. I noticed the presence of a world-writable directory and so I tightened the security of some of the default mount points by putting the following in /etc/rc.local:

mount -o remount,nodev,nosuid /etc/network
mount -o remount,nodev,nosuid /lib/modules
chmod 755 /etc/network
exit 0

Hardware setup My OS drive (/dev/sda) is a small SSD so that the GnuBee can run silently when the spinning disks aren't needed. To hold the backup data on the other hand, I got three 4-TB drives drives which I setup in a RAID-5 array. If the data were valuable, I'd use RAID-6 instead since it can survive two drives failing at the same time, but in this case since it's only holding backups, I'd have to lose the original machine at the same time as two of the 3 drives, a very unlikely scenario. I created new gpt partition tables on /dev/sdb, /dev/sdbc, /dev/sdd and used fdisk to create a single partition of type 29 (Linux RAID) on each of them. Then I created the RAID array:

mdadm /dev/md127 --create -n 3 --level=raid5 -a /dev/sdb1 /dev/sdc1 /dev/sdd1

and waited more than 24 hours for that operation to finish. Next, I formatted the array:

mkfs.ext4 -m 0 /dev/md127

and added the following to /etc/fstab:

/dev/md127 /mnt/data/ ext4 noatime,nodiratime 0 2

To reduce unnecessary noise and reduce power consumption, I also installed hdparm:

apt install hdparm

and configured all spinning drives to spin down after being idle for 10 minutes by putting the following in /etc/hdparm.conf:

/dev/sdb  
       spindown_time = 120
 
/dev/sdc  
       spindown_time = 120
 
/dev/sdd  
       spindown_time = 120

and then reloaded the configuration:

 /usr/lib/pm-utils/power.d/95hdparm-apm resume

Finally I setup smartmontools by putting the following in /etc/smartd.conf:

/dev/sda -a -o on -S on -s (S/../.././02 L/../../6/03)
/dev/sdb -a -o on -S on -s (S/../.././02 L/../../6/03)
/dev/sdc -a -o on -S on -s (S/../.././02 L/../../6/03)
/dev/sdd -a -o on -S on -s (S/../.././02 L/../../6/03)

and restarting the daemon:

systemctl restart smartd.service

Backup setup I started by using duplicity since I have been using that tool for many years, but a 190GB backup took around 15 hours on the GnuBee with gigabit ethernet. After a friend suggested it, I took a look at restic and I have to say that I am impressed. The same backup finished in about half the time.

User and ssh setup After hardening the ssh setup as I usually do, I created a user account for each machine needing to backup onto the GnuBee:

adduser machine1
adduser machine1 sshuser
adduser machine1 sftponly
chsh machine1 -s /bin/false

and then matching directories under /mnt/data/home/:

mkdir /mnt/data/home/machine1
chown machine1:machine1 /mnt/data/home/machine1
chmod 700 /mnt/data/home/machine1

Then I created a custom ssh key for each machine:

ssh-keygen -f /root/.ssh/foobar_backups -t ed25519

and placed it in /home/machine1/.ssh/authorized_keys on the GnuBee. On each machine, I added the following to /root/.ssh/config:

Host foobar.local
    User machine1
    Compression no
    Ciphers aes128-ctr
    IdentityFile /root/backup/foobar_backups
    IdentitiesOnly yes
    ServerAliveInterval 60
    ServerAliveCountMax 240

The reason for setting the ssh cipher and disabling compression is to speed up the ssh connection as much as possible given that the GnuBee has avery small RAM bandwidth. Another performance-related change I made on the GnuBee was switching to the internal sftp server by putting the following in /etc/ssh/sshd_config:

Subsystem      sftp    internal-sftp

Restic script After reading through the excellent restic documentation, I wrote the following backup script, based on my old duplicity script, to reuse on all of my computers:

# Configure for each host
PASSWORD="XXXX"  # use  pwgen -s 64  to generate a good random password
BACKUP_HOME="/root/backup"
REMOTE_URL="sftp:foobar.local:"
RETENTION_POLICY="--keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 2"
# Internal variables
SSH_IDENTITY="IdentityFile=$BACKUP_HOME/foobar_backups"
EXCLUDE_FILE="$BACKUP_HOME/exclude"
PKG_FILE="$BACKUP_HOME/dpkg-selections"
PARTITION_FILE="$BACKUP_HOME/partitions"
# If the list of files has been requested, only do that
if [ "$1" = "--list-current-files" ]; then
    RESTIC_PASSWORD=$PASSWORD restic --quiet -r $REMOTE_URL ls latest
    exit 0
# Show list of available snapshots
elif [ "$1" = "--list-snapshots" ]; then
    RESTIC_PASSWORD=$GPG_PASSWORD restic --quiet -r $REMOTE_URL snapshots
    exit 0
# Restore the given file
elif [ "$1" = "--file-to-restore" ]; then
    if [ "$2" = "" ]; then
        echo "You must specify a file to restore"
        exit 2
    fi
    RESTORE_DIR="$(mktemp -d ./restored_XXXXXXXX)"
    RESTIC_PASSWORD=$PASSWORD restic --quiet -r $REMOTE_URL restore latest --target "$RESTORE_DIR" --include "$2"   exit 1
    echo "$2 was restored to $RESTORE_DIR"
    exit 0
# Delete old backups
elif [ "$1" = "--prune" ]; then
    # Expire old backups
    RESTIC_PASSWORD=$PASSWORD restic --quiet -r $REMOTE_URL forget $RETENTION_POLICY
    # Delete files which are no longer necessary (slow)
    RESTIC_PASSWORD=$PASSWORD restic --quiet -r $REMOTE_URL prune
    exit 0
# Catch invalid arguments
elif [ "$1" != "" ]; then
    echo "Invalid argument: $1"
    exit 1
fi
# Check the integrity of existing backups
RESTIC_PASSWORD=$PASSWORD restic --quiet -r $REMOTE_URL check   exit 1
# Dump list of Debian packages
dpkg --get-selections > $PKG_FILE
# Dump partition tables from harddrives
/sbin/fdisk -l /dev/sda > $PARTITION_FILE
/sbin/fdisk -l /dev/sdb > $PARTITION_FILE
# Do the actual backup
RESTIC_PASSWORD=$PASSWORD restic --quiet --cleanup-cache -r $REMOTE_URL backup / --exclude-file $EXCLUDE_FILE

I run it with the following cronjob in /etc/cron.d/backups:

30 8 * * *    root  ionice nice nocache /root/backup/backup-machine1-to-foobar
30 2 * * Sun  root  ionice nice nocache /root/backup/backup-machine1-to-foobar --prune

in a way that doesn't impact the rest of the system too much. Finally, I printed a copy of each of my backup script, using enscript, to stash in a safe place:

enscript --highlight=bash --style=emacs --output=- backup-machine1-to-foobar   ps2pdf - > foobar.pdf

This is actually a pretty important step since without the password, you won't be able to decrypt and restore what's on the GnuBee.

Next.

Previous.