Search Results: "Elana Hashman"

29 May 2023

Louis-Philippe V ronneau: Python 3.11, pip and (breaking) system packages

As we get closer to Debian Bookworm's release, I thought I'd share one change in Python 3.11 that will surely affect many people. Python 3.11 implements the new PEP 668, Marking Python base environments as externally managed 1. If you use pip regularly on Debian, it's likely you'll eventually hit the externally-managed-environment error: 
error: externally-managed-environment
  This environment is externally managed
 > To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    See /usr/share/doc/python3.11/README.venv for more information.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
With this PEP, Python tools can now distinguish between packages that have been installed by the user with a tool like pip and ones installed using a distribution's package manager, like apt. This is generally great news: it was previously too easy to break a system by mixing the two types of packages. This PEP will simplify our role as a distribution, as well as improve the overall Python user experience in Debian. Sadly, it's also likely this change will break some of your scripts, especially CI that (legitimately) install packages via pip alongside system packages. For example, I use the following gitlab-ci snippet to make sure my PRs don't break my build process2: 
build:flit:
  stage: build
  script:
  - apt-get update && apt-get install -y flit python3-pip
  - FLIT_ROOT_INSTALL=1 flit install
  - metalfinder --help
With Python 3.11, this snippet will error out, as pip will refuse to install packages alongside the system's. The fix is to tell pip it's OK to "break" your system packages, either using the --break-system-packages parameter, or the PIP_BREAK_SYSTEM_PACKAGES=1 environment variable3. This, of course, is not something you should be using in production to restore the old behavior! The "proper" way to fix this issue, as the externally-managed-environment error message aptly (har har) informs you, is to use virtual environments. Happy hacking!
  1. Kudos to our own Matthias Klose, Stefano Rivera and Elana Hashman, who worked on designing and implementing this PEP!
  2. Which is something that bit me before... You push some changes to your git repository, everything seems fine and all the tests pass, so you merge it and make a new git tag. When the time comes to build and upload this tag to PyPi, you find out some minor thing broke your build system (which you weren't testing) and you have to scramble to make a point-release to fix the issue. Sad!
  3. Don't go searching for this environment variable in pip's code though, as you won't find it! All of pip's command line options can be passed as env vars using the PIP_<UPPER_LONG_NAME> format. Useful for tools that use pip indirectly, like flit.

18 May 2022

Louis-Philippe V ronneau: Clojure Team 2022 Sprint Report

This is the report for the Debian Clojure Team remote sprint that took place on May 13-14th. Looking at my previous blog entries, this was my first Debian sprint since July 2020! Crazy how fast time flies... Many thanks to those who participated, namely: Sadly, Utkarsh Gupta although having planned on participating ended up not being able to and worked on DebConf Bursary paperwork instead. rlb Rob mostly worked on creating a dh-clojure tool to help make packaging Clojure libraries easier. At the moment, most of the packaging is done manually, by invoking build tools by hand. Having a tool to automate many of the steps required to build Clojure packages would go a long way in making them more uniform. His work (although still very much a WIP) can be found here: https://salsa.debian.org/rlb/dh-clojure/ ehashman Elana: lavamind It was J r me's first time working on Clojure packages, and things went great! During the sprint, he: allentiak Leandro joined us on Saturday, since he couldn't get off work on Friday. He mostly continued working on replacing our in-house scripts for /usr/bin/clojure by upstream's, a task he had already started during GSoC 2021. Sadly, none of us were familiar with Debian's mechanism for alternatives. If you (yes you, dear reader) are familiar with it, I'm sure he would warmly welcome feedback on his development branch. pollo As for me, I: Overall, it was quite a productive sprint! Thanks to Debian for sponsoring our food during the sprint. It was nice to be able to concentrate on fixing things instead of making food :) Here's a bonus picture of the nice sushi platter I ended up getting for dinner on Saturday night: Picture of a sushi platter

23 August 2021

Leandro Doctors: Clojure CLI Tools in Debian - GSoC 2021 Final Report

NOTE: this blog post is based on my "Clojure CLI Tools in Debian" GSoC 2021 project Final Report.

IntroMy name is Leandro Doctors ( allentiak on IRC), and I ve been the GSoC intern working with the Debian Clojure Team during 2021. This is my final report. You can also check my original proposal and my first report.

SummaryWhereas the raw data may not sound by itself very positive, my personal conclusion is. This is, whereas I didn t fully finish the required deliverables envisioned in my original proposal, I do feel I am much closer to, eventually, becoming a Debian Developer. So, by all means, I consider this project has had a positive outcome.

ProjectThe goal of the Clojure Build Tools in Debian project was to provide Clojure Debian users with some of the latest advanced build tools and libraries the upstream Clojure developers have been lately working on. These include tools.deps.alpha, a library for dependency graph resolution and classpath building, and the CLI tool clj, for REPL interaction. If time permitted, I was also to improve the quality of both new and existing Clojure packages, and the overall Debian Clojure packaging process. My mentor was Louis-Louis-Philippe V ronneau, and my co-mentor was Utkarsh Gupta.

MotivationWhy this project? On the one side, if you re a Clojure lover like me, you may have noticed that the Clojure experience in Debian is, as of mid 2021, well... still quiet limited. Additionally, this project aligned with my own background in Free Software community building and my research interest in Peer Production.When I mention how limited today s Clojure experience in Debian is, I can see two reasons for this, deeply intertwined. The first one is that there currently aren t many Clojure-specific packaging tools in Debian (such as a clojure-debian-helper). The second reason for which we only currently have a suboptimal Clojure experience in Debian, and probably the root of the previous one, is that many core build tools and libraries for the language have not simply been packaged yet. My project aimed to attack that seemingly root cause.As I said, another reason for me choosing this project is my own experience as the Co-founder and Leader of, probably, the first Free Software Community experience in my hometown of San Juan, Argentina. That interest in Free Software evolved in a first PhD attempt in what is now known as the field of Peer Production. A subject that has lived within me as a research interest during my day job at a University.Being a Clojure fan, it felt only logical combining all those interests somehow. And this project seemed like the ideal combination.

The Debian Clojure TeamI ve been working with a small, yet very warm team. The current incarnation of the Debian Clojure Team exists thanks to the hard work of three people.Elana Hashman (aka the Clojure necromancer ), revived the team around three years ago. Later on, the team gained the invaluable presence of Louis-Philippe V ronneau and Utkarsh Gupta (my mentor and co-mentor, respectively).Together, these Three Musketeers have maintained the team alive, allowing us, Debian users, to enjoy Clojure.

StatusDuring the first part of my project, I mainly worked on learning the basics of Debian packaging, and got my first package uploaded. I have to thank Louis-Philippe, Utkarsh, and Elana for their immense patience and support during that part, as it took me quite some effort grasping the basics of Debian packaging.During the second part of my project, I worked on my last packages, and almost completed the originally required scope of the project. I only have to finish working on the transition from the currently provided set of packages (based on a Debian-specific clojure runner) to the newly provided upstream clojure and clj runners.Unfortunately, I didn t have much time left to start working on the opportunities for improvement already identified by the Debian Clojure Team originally outlined in my proposal. Whereas I did update one older Clojure package not built using leiningen (tools-data-xml-clojure), I didn t write any Lintian tags to make Clojure packaging in Debian more robust, nor worked towards the automation of Clojure unit tests in autopkgtests via autodep8.

Deliverables: Data vs. ConclusionsIf we are to talk about deliverables, we should start with the data. According to my original proposal, I was required to provide both new and updated Clojure packages accepted into Debian unstable , and updated Clojure packaging documentation. Additionally, if time permitted, I was to also provide new Clojure Lintian tags merged by the Lintian maintainers, and new Clojure autodep8 scripts merged by the autodep8 maintainers. Whereas I partially accomplished both required tasks, I didn t manage to start working on any of the optional deliverables.When looked in isolation, those numbers may look somewhat disappointing for some people. However, I can draw a much more positive conclusion. Why?Firstly, GSoC is supposed to be a learning experience. Moreover, as I said in my original proposal, I approach[ed] this project as a great opportunity to, finally, start my journey towards becoming a Debian Developer . In that sense, I consider the time invested into this project fruitful. In this way, I have learned the basics of packaging, how to interact with the Debian Clojure Team, and and already got my first packages accepted. Plus, I m looking forward to continuing to work with the Debian Clojure Team so I can attain the original scope of the project. Therefore, all things considered, I can consider this experience as a moderate success.

Lessons LearnedTechnically speaking, if I have learned one thing during these weeks, is that packaging, although easy to be underestimated, is by no means a trivial process. As any Debian Developer surely knows, the onboarding process can take some time. Plus, what is easy for some people, can be difficult for others. In my case, this was quite evident. Whereas I can speak several languages and learning new ones takes me little effort, grasping the basics of packaging took me (literally!) blood, sweat and tears. Indeed, the packaging learning curve was quite steep for me.That being said, I did learn a thing or two about packaging. So, if I managed to get here, I m sure many others can. It may take them more or less time than what it took me, but learning (at least the basics) of packaging is an achievable goal.Technical skill learning aside, I value very highly the non-technical skills I have so far improved during this project.For instance, I also learned that it can take some time to adapt to real-time online communication. Before this project, remote working meant either exchanging emails or getting into video or audio calls, with a low emphasis on chat-based interaction. Early on, I realized that the Debian Clojure Team interacts almost exclusively via, well... chat! And those two approaches are very different indeed. It has taken me some time to adjust, but I ve improved greatly in this aspect as well.Finally, improving my time management skills has been also a key part of this process. Whereas I had already been working remotely for over a year and a half already, my day job is not so interaction-dependent as this project (specially in the beginning). So it took me some time to adapt to this way of working, and to plan my workload so I could use those waiting moments to advance in other parts of the project. Still a lot to improve here, but improving nevertheless.

AcknowledgmentsI first have to thank upstream. More specifically, one of the upstream developers of the clojure-tools, Alex Miller. Everytime I needed specific information on what do specific parts of the Clojure CLI tools s codebase do, tools.deps.alpha do, he popped up a reply in a matter of hours. He has shown genuine interest in the success of is project during by carefully replying to my emails with detailed explanations of code intent and form, both in private and in public conversations. Thank you for all that, Alex!Let s move on to the Debian Clojure Team.First, Elana. I thank Elana for her initial openness when I first contacted her about this idea. It was *her* who initially contacted Louis-Philippe so he would become my mentor. I wouldn t have started to work on this project if it wasn t for her. Plus, she provided quite a piece of advise in more that one ocassion. So, thank you very much for all that, Elana!I also thank Utkarsh, my co-mentor, for his overall technical advise. And a special mention to his initial help to setup my Matrix client for OFTC chat. At that moment, it was *him* who took the time to help me in real time so I could solve that problem. So, thank you very much for all that, Utkarsh!I finally have to thank Louis-Philippe, my mentor, for his patient guidance during the whole process. His dedication and hard work has been *instrumental* for my progress. And a special mention for his tolerance with respect to some unforeseen personal circumstances I had to endure during the first weeks. When one is playing the newbie, times abound when one depends on other people s feedback. And Debian is made of volunteers, who have a life outside it. Every time I asked, Louis-Philippe was there. I wouldn t have gotten here if it wasn t for him. So, thank you *so* much for all that, Louis-Philippe!

Final WordsI would like to close this report with a reflection.I have been using Debian for many, many years now, and I had been looking for a way to contribute back to the project for some time already. I even did some work on a non-packaging Debian project. That being said, I never managed to deliver much, really.So, the very existence of outreach programs as this one is, in my humble opinion, crucial. In my case, the funding I got through the GSoC program was instrumental in being able to allocate time for this endeavor, and to finally get started contributing to Debian. Plus, it has had a very positive impact on me; in many ways, some of which I am only starting to discover now that the project is ending.When I put things into perspective, this project is very important for me. Actually, it is nothing but the first step within a long-term journey: becoming a Debian member. Hopefully, I would like to be able to apply for Debian membership by the end of this year.

Questions?Thank you very much for your time reading this! I look forward to hearing (or reading) your feedback. Please come and meet with the Debian Clojure Team Moreover, I will be in the Clojure BoF on DebConf2021. Moreover, do not hesitate to send me an email.

Data

Task Status
  • Required Tasks:
    • T1: Setting up a full Debian packaging development environment and learning the basics of Debian packaging.
      • Successfully completed the first part during the Application period.
      • Successfully completed the second part during the Coding periods.
    • T2: Identifying and packaging the missing dependencies to package clojure-cli.
      • Successfully completed as of the end of Coding II.
    • T3: Packaging clojure-cli.
      • 90% done as of the end of Coding II.
    • T4: Updating clojure to use clojure-cli.
      • To be completed after GSoC.
    • T5: Updating the Clojure Packaging Guide with information on how to use the new clojure-cli scripts.
      • Improved existing documentation. To be completed after GSoC.
  • Optional Tasks:
    • T6: Writing Lintian tags to make Clojure packaging in Debian more robust.
      • To be completed after GSoC.
    • T7: Working to automate Clojure unit testing in autopkgtests using autodep8.
      • To be completed after GSoC.
    • T8: Updating older Clojure packages not built using leiningen or clojure-cli.
      • To be completed after GSoC.

Packages
  1. Updated packages:
  2. New packages:
    • tools-gitlibs-clojure -- Clojure API for programatically accessing git libraries. ITP: #905543 in NEW.
    • ITP: tools-deps-alpha-clojure -- functional API for dependency management and classpath creation https://bugs.debian.org/891136 Needs to be uploaded by Louis-Philippe.
  3. In-Progress packages:
    • ITP: clojure-cli -- upstream CLI entrypoints for Clojure https://bugs.debian.org/891141 90% done - Package completed. I only need to finish implementing the transition from existing clojure scripts. To be completed after GSoC.
  4. Opened ITPs:
  5. Reported bugs

Other
  1. Interactions with upstream in the Debian-Clojure mailing list:
    • Many productive interactions with one of the upstream developers, Alex Miller (June, August).
  2. Wiki page:
    1. https://wiki.debian.org/Clojure/Goals/ClojureCLIToolsInDebian

13 August 2021

Leandro Doctors: Clojure CLI Tools in Debian - GSoC 2021 Partial Evaluation Report

NOTE: this blog post is based on my "Clojure CLI Tools in Debian" GSoC 2021 project Partial Evaluation Report.Hi, everybody!For those who don't know me, my name is Leandro Doctors (allentiak on IRC), and I'm the Debian Clojure Team's GSoC 2021 intern. My mentor is Louis-Philippe V ronneau[*] (pollo on IRC). My co-mentor is Utkarsh Gupta (utkarsh2102 on IRC). My 'no-mentor' :) is Elana Hashman (ehashman on IRC).In this message, I will summarize what I've been up to during the Coding I period (June 7th - July 16th).
TL;DR: my updated data-xml-clojure package was accepted[1] in experimental on Wednesday night :-)
[1] https://tracker.debian.org/news/1244465/accepted-data-xml-clojure-020alpha6-1-source-into-experimental/Now, let's tell the full story.During the first days of the "Coding I" phase, I did some research on the clj dependencies. And after the precious feedback from Louis-Philippe, Elana, Utkarsh, and Alex Miller (the upstream developer of clj, among many other libraries), I came up with the following packaging strategy.
  1. update org.clojure:data.xml (data-xml-clojure) to 0.2.0-beta6.
  2. package org.clojure:tools.gitlibs (tools-gitlibs-clojure).
  3. consider updating libjsch-agent-proxy-java, jgit, libtools-cli-clojure (all of them already in Debian).
  4. consider packaging com.cognitec.aws:* packages.
According to my original proposal, I should have completed all four tasks during Coding I. Looking back, the main lesson from these past weeks is a known classic: my timeline was too optimistic: I definitely underestimated the difficulty of the packaging process. Out of the four tasks, I only finished the first one.There were many challenges I had to overcome in order to update the library from version 0.0.8 to 0.2.0-alpha6:This what I did:First, I patched the source code so:Then, I completely overhauled the packaging code (this is, what goes inside debian/).All this improved the quality of the package.I also improved the Clojure Packaging Tutorial[2] to make the process easier to follow.[2] https://wiki.debian.org/Clojure/PackagingTutorialLooking back, it is almost as if I had started packaging the library from scratch...But, more that what I produced, I think the most important part of all this is what I learned during these weeks.As it was the first time I ever packaged anything in Debian, I had to learn the basics, bump by bump. (And, oh my, I surely did bump quite a few times!)[2] https://wiki.debian.org/Clojure/PackagingTutorial[3] https://wiki.debian.org/sbuild[4] https://manpages.debian.org/unstable/git-buildpackage/gbp.1.en.html[5] https://wiki.debian.org/UsingQuiltDefinitely, all this was worth it. After all, my updated data-xml-clojure package was accepted[1] in experimental on Wednesday :-)[1] https://tracker.debian.org/news/1244465/accepted-data-xml-clojure-020alpha6-1-source-into-experimental/Now that I've learned the basics of packaging bumped enough to get my package accepted, I'm hopeful I can ramp up and catch up with (at least most of) my original schedule during Coding II.tools-gitlibs-clojure, you're next :-)Thank you very much Louis-Philippe, Elana, and Utkarsh (plus a special mention to Alex) for your precious support during the last few weeks!

Leandro Doctors: Clojure CLI Tools in Debian - GSoC 2021 Partial Evaluation Report

NOTE: this blog post is based on my "Clojure CLI Tools in Debian" GSoC 2021 project Partial Evaluation Report.Hi, everybody!For those who don't know me, my name is Leandro Doctors (allentiak on IRC), and I'm the Debian Clojure Team's GSoC 2021 intern. My mentor is Louis-Philippe V ronneau[*] (pollo on IRC). My co-mentor is Utkarsh Gupta (utkarsh2102 on IRC). My 'no-mentor' :) is Elana Hashman (ehashman on IRC).In this message, I will summarize what I've been up to during the Coding I period (June 7th - July 16th).
TL;DR: my updated data-xml-clojure package was accepted[1] in experimental on Wednesday night :-)
[1] https://tracker.debian.org/news/1244465/accepted-data-xml-clojure-020alpha6-1-source-into-experimental/Now, let's tell the full story.During the first days of the "Coding I" phase, I did some research on the clj dependencies. And after the precious feedback from Louis-Philippe, Elana, Utkarsh, and Alex Miller (the upstream developer of clj, among many other libraries), I came up with the following packaging strategy.
  1. update org.clojure:data.xml (data-xml-clojure) to 0.2.0-beta6.
  2. package org.clojure:tools.gitlibs (tools-gitlibs-clojure).
  3. consider updating libjsch-agent-proxy-java, jgit, libtools-cli-clojure (all of them already in Debian).
  4. consider packaging com.cognitec.aws:* packages.
According to my original proposal, I should have completed all four tasks during Coding I. Looking back, the main lesson from these past weeks is a known classic: my timeline was too optimistic: I definitely underestimated the difficulty of the packaging process. Out of the four tasks, I only finished the first one.There were many challenges I had to overcome in order to update the library from version 0.0.8 to 0.2.0-alpha6:This what I did:First, I patched the source code so:Then, I completely overhauled the packaging code (this is, what goes inside debian/).All this improved the quality of the package.I also improved the Clojure Packaging Tutorial[2] to make the process easier to follow.[2] https://wiki.debian.org/Clojure/PackagingTutorialLooking back, it is almost as if I had started packaging the library from scratch...But, more that what I produced, I think the most important part of all this is what I learned during these weeks.As it was the first time I ever packaged anything in Debian, I had to learn the basics, bump by bump. (And, oh my, I surely did bump quite a few times!)[2] https://wiki.debian.org/Clojure/PackagingTutorial[3] https://wiki.debian.org/sbuild[4] https://manpages.debian.org/unstable/git-buildpackage/gbp.1.en.html[5] https://wiki.debian.org/UsingQuiltDefinitely, all this was worth it. After all, my updated data-xml-clojure package was accepted[1] in experimental on Wednesday :-)[1] https://tracker.debian.org/news/1244465/accepted-data-xml-clojure-020alpha6-1-source-into-experimental/Now that I've learned the basics of packaging bumped enough to get my package accepted, I'm hopeful I can ramp up and catch up with (at least most of) my original schedule during Coding II.tools-gitlibs-clojure, you're next :-)Thank you very much Louis-Philippe, Elana, and Utkarsh (plus a special mention to Alex) for your precious support during the last few weeks!

17 June 2021

Elana Hashman: I'm hosting a Bug Scrub for Kubernetes SIG Node

It's been a long while since I last hosted a BSP, but 'tis the season. Kubernetes SIG Node will be holding a bug scrub on June 24-25, and this is a great opportunity for you to get involved if you're interested in contributing to Kubernetes or SIG Node! We will be hosting a global event with region captains for all timezones. I am one of the NASA captains (~17:00-01:00 UTC) and I'll be leading the kickoff. We will be working on Slack and Zoom. I hope you'll be able to drop in! Details I'm an existing contributor, what should I work on? Work on triaging and closing SIG Node bugs. We have a lot of bugs!! The goal of our event is to categorize, clean up, and resolve some of the 450+ issues in k/k for SIG Node. Check out the event docs for more instructions. I'm a new contributor and want to join but I have no idea what I'm doing! At some point, that was all of us! This is a great opportunity to get involved if you've never contributed to Kubernetes. We'll have dedicated mentors available to coordinate and help out new contributors. If you've never contributed to Kubernetes before, I recommend you check out the Getting Started and Contributor Guide resources in advance of the event. You will want to ensure you've signed the contributor license agreement (CLA). Remember, you don't have to code to make valuable contributions! Triaging the bug tracker is a great example of this. See you there! Happy hacking.

26 March 2021

Daniel Lange: The Stallman wars

So, 2021 isn't bad enough yet, but don't despair, people are working to fix that:

Welcome to the Stallman wars Team Cancel: https://rms-open-letter.github.io/ (repo) Team Support: https://rms-support-letter.github.io/ (repo) Current stats are: 

Team Cancel:  3028 signers from 1413 individual commit authors
Team Support: 6249 signers from 5018 individual commit authors
Git shortlog (Top 10): 
rms_cancel.git (Last update: 2021-04-07 15:42:33 (UTC))
  1228  Neil McGovern
   251  Joan Touzet
    86  Elana Hashman
    71  Molly de Blanc
    36  Shauna
    19  Juke
    18  Stefano Zacchiroli
    17  Alexey Mirages
    16  Devin Halladay
    14  Nader Jafari
rms_support.git (Last update: 2021-04-12 09:25:53 (UTC))
  1678  shenlebantongying
  1564  nukeop
  1550  Ivanq
   826  Victor
   746  Job Bautista
   123  nekonee
    61  Victor Gridnevsky
    38  Patrick Spek
    25  Borys Kabakov
    17  KIM Taeyeob
(last updated 2021-04-12 09:26:15 (UTC)) Technical info:
Signers are counted from their "Signed / Individuals" sections. Commits are counted with git shortlog -s.
Team Cancel also has organizational signatures with Mozilla, Suse and X.Org being among the notable signatories. Debian is in the process of running a GR to join (or not join) that list. The 16 original signers of the Cancel petition are added in their count. Neil McGovern, Juke and shenlebantongying need .mailmap support as they have committed with different names. Further reading:

9 January 2021

Louis-Philippe V ronneau: puppetserver 6: a Debian packaging post-mortem

I have been a Puppet user for a couple of years now, first at work, and eventually for my personal servers and computers. Although it can have a steep learning curve, I find Puppet both nimble and very powerful. I also prefer it to Ansible for its speed and the agent-server model it uses. Sadly, Puppet Labs hasn't been the most supportive upstream and tends to move pretty fast. Major versions rarely last for a whole Debian Stable release and the upstream .deb packages are full of vendored libraries.1 Since 2017, Apollon Oikonomopoulos has been the one doing most of the work on Puppet in Debian. Sadly, he's had less time for that lately and with Puppet 5 being deprecated in January 2021, Thomas Goirand, Utkarsh Gupta and I have been trying to package Puppet 6 in Debian for the last 6 months. With Puppet 6, the old ruby Puppet server using Passenger is not supported anymore and has been replaced by puppetserver, written in Clojure and running on the JVM. That's quite a large change and although puppetserver does reuse some of the Clojure libraries puppetdb (already in Debian) uses, packaging it meant quite a lot of work. Work in the Clojure team As part of my efforts to package puppetserver, I had the pleasure to join the Clojure team and learn a lot about the Clojure ecosystem. As I mentioned earlier, a lot of the Clojure dependencies needed for puppetserver were already in the archive. Unfortunately, when Apollon Oikonomopoulos packaged them, the leiningen build tool hadn't been packaged yet. This meant I had to rebuild a lot of packages, on top of packaging some new ones. Since then, thanks to the efforts of Elana Hashman, leiningen has been packaged and lets us run the upstream testsuites and create .jar artifacts closer to those upstream releases. During my work on puppetserver, I worked on the following packages:
List of packages
  • backport9
  • bidi-clojure
  • clj-digest-clojure
  • clj-helper
  • clj-time-clojure
  • clj-yaml-clojure
  • cljx-clojure
  • core-async-clojure
  • core-cache-clojure
  • core-match-clojure
  • cpath-clojure
  • crypto-equality-clojure
  • crypto-random-clojure
  • data-csv-clojure
  • data-json-clojure
  • data-priority-map-clojure
  • java-classpath-clojure
  • jnr-constants
  • jnr-enxio
  • jruby
  • jruby-utils-clojure
  • kitchensink-clojure
  • lazymap-clojure
  • liberator-clojure
  • ordered-clojure
  • pathetic-clojure
  • potemkin-clojure
  • prismatic-plumbing-clojure
  • prismatic-schema-clojure
  • puppetlabs-http-client-clojure
  • puppetlabs-i18n-clojure
  • puppetlabs-ring-middleware-clojure
  • puppetserver
  • raynes-fs-clojure
  • riddley-clojure
  • ring-basic-authentication-clojure
  • ring-clojure
  • ring-codec-clojure
  • shell-utils-clojure
  • ssl-utils-clojure
  • test-check-clojure
  • tools-analyzer-clojure
  • tools-analyzer-jvm-clojure
  • tools-cli-clojure
  • tools-reader-clojure
  • trapperkeeper-authorization-clojure
  • trapperkeeper-clojure
  • trapperkeeper-filesystem-watcher-clojure
  • trapperkeeper-metrics-clojure
  • trapperkeeper-scheduler-clojure
  • trapperkeeper-webserver-jetty9-clojure
  • url-clojure
  • useful-clojure
  • watchtower-clojure
If you want to learn more about packaging Clojure libraries and applications, I rewrote the Debian Clojure packaging tutorial and added a section about the quirks of using leiningen without a dedicated dh_lein tool. Work left to get puppetserver 6 in the archive Unfortunately, I was not able to finish the puppetserver 6 packaging work. It is thus unlikely it will make it in Debian Bullseye. If the issues described below are fixed, it would be possible to to package puppetserver in bullseye-backports though. So what's left? jruby Although I tried my best (kudos to Utkarsh Gupta and Thomas Goirand for the help), jruby in Debian is still broken. It does build properly, but the testsuite fails with multiple errors: jruby testsuite failures aside, I have not been able to use the jruby.deb the package currently builds in jruby-utils-clojure (testsuite failure). I had the same exact failure with the (more broken) jruby version that is currently in the archive, which leads me to think this is a LOAD_PATH issue in jruby-utils-clojure. More on that below. To try to bypass these issues, I tried to vendor jruby into jruby-utils-clojure. At first I understood vendoring meant including upstream pre-built artifacts (jruby-complete.jar) and shipping them directly. After talking with people on the #debian-mentors and #debian-ftp IRC channels, I now understand why this isn't a good idea (and why it's not permitted in Debian). Many thanks to the people who were patient and kind enough to discuss this with me and give me alternatives. As far as I now understand it, vendoring in Debian means "to have an embedded copy of the source code in another package". Code shipped that way still needs to be built from source. This means we need to build jruby ourselves, one way or another. Vendoring jruby in another package thus isn't terribly helpful. If fixing jruby the proper way isn't possible, I would suggest trying to build the package using embedded code copies of the external libraries jruby needs to build, instead of trying to use the Debian libraries.2 This should make it easier to replicate what upstream does and to have a final .jar that can be used. jruby-utils-clojure This package is a first-level dependency for puppetserver and is the glue between jruby and puppetserver. It builds fine, but the testsuite fails when using the Debian jruby package. I think the problem is caused by a jruby LOAD_PATH issue. The Debian jruby package plays with the LOAD_PATH a little to try use Debian packages instead of downloading gems from the web, as upstream jruby does. This seems to clash with the gem-home, gem-path, and jruby-load-path variables in the jruby-utils-clojure package. The testsuite plays around with these variables and some Ruby libraries can't be found. I tried to fix this, but failed. Using the upstream jruby-complete.jar instead of the Debian jruby package, the testsuite passes fine. This package could clearly be uploaded to NEW right now by ignoring the testsuite failures (we're just packaging static .clj source files in the proper location in a .jar). puppetserver jruby issues aside, packaging puppetserver itself is 80% done. Using the upstream jruby-complete.jar artifact, the testsuite fails with a weird Clojure error I'm not sure I understand, but I haven't debugged it for very long. Upstream uses git submodules to vendor puppet (agent), hiera (3), facter and puppet-resource-api for the testsuite to run properly. I haven't touched that, but I believe we can either: Without the testsuite actually running, it's hard to know what files are needed in those packages. What now Puppet 5 is now deprecated. If you or your organisation cares about Puppet in Debian,3 puppetserver really isn't far away from making it in the archive. Very talented Debian Developers are always eager to work on these issues and can be contracted for very reasonable rates. If you're interested in contracting someone to help iron out the last issues, don't hesitate to reach out via one of the following: As for I, I'm happy to say I got a new contract and will go back to teaching Economics for the Winter 2021 session. I might help out with some general Debian packaging work from time to time, but it'll be as a hobby instead of a job. Thanks The work I did during the last 6 weeks would be not have been possible without the support of the Wikimedia Foundation, who were gracious enough to contract me. My particular thanks to Faidon Liambotis, Moritz M hlenhoff and John Bond. Many, many thanks to Rob Browning, Thomas Goirand, Elana Hashman, Utkarsh Gupta and Apollon Oikonomopoulos for their direct and indirect help, without which all of this wouldn't have been possible.
  1. For example, the upstream package for the Puppet Agent vendors OpenSSL.
  2. One of the problems of using Ruby libraries already packaged in Debian is that jruby currently only supports Ruby 2.5. Ruby libraries in Debian are currently expected to work with Ruby 2.7, with the transition to Ruby 3.0 planned after the Bullseye release.
  3. If you run Puppet, you clearly should care: the .deb packages upstream publishes really aren't great and I would not recommend using them.

17 December 2020

Elana Hashman: Easy risotto

I have a secret: risotto doesn't have to involve standing over a pot, stirring for 30 minutes. There is a better way. I've made risotto using the "stir 5ever" method and frankly this one is half the work and just as good. I took inspiration from Kenji Lopez-Alt, who describes how you can make risotto with only a few minutes of stirring at the end! Yes, this is a secret too good to keep to myself. The linked recipe is too rich for a weeknight dinner, at least for my tastes, so I've included my typical modifications below, which omits the heavy cream. I usually make mushroom risotto, because all the ingredients are long-lived pantry ingredients, which means I can make this on a whim without having to grocery shop! Mushroom risotto Serves 3 as a main course. Active time: 25m. Total time: 45m. Ingredients: Recipe:
  1. Leave mushrooms to rehydrate in 1 cup of hot water for at least 10 minutes.
  2. Set a wide saucepan (about 3.5qts) or skillet on medium heat.
  3. Mix stock and wine for a total of 4 cups. You can used homemade stock (the best), boxed, or reconstitute from bouillon (my usual). If you use bouillon, I find you get the best flavour if you use a mix, so I tend to use half "no-chicken" and half "organic chicken" of the Better than Bouillon brand. Let this cool if you used boiling water; you don't want it to be too hot.
  4. Mix the rice and stock in a large bowl, agitating it to release the starch. Reserve stock and set the rice in a strainer to drain.
  5. Small dice an onion. Add olive oil and butter to the pan, and then add the onion. Cook the onion until it's translucent, but don't brown it.
  6. Increase the heat to high and add the rice to the pan to toast until it becomes a little bit brown, stirring occasionally, about 3-4 minutes. Set a timer so it doesn't burn
  7. Meanwhile, drain the mushrooms, making sure you reserve the broth. Chop mushrooms and mince the garlic. Once the rice is toasted, add mushrooms and garlic and stir.
  8. Stir the reserved stock. Add all of the mushroom liquid and all but 1 cup of the starchy stock to the pan. Bring to a boil.
  9. Once it has reached a boil, cover, reduce to the lowest heat, and allow the rice to simmer undisturbed for 10 minutes. In the meantime, you can grate the cheese.
  10. After 10 minutes, stir the rice once, shake the pan to ensure it is level, and return to the heat for 10 more minutes. Sit back, relax, and bask in the very little stirring you are doing.
  11. At this point, the rice should be nearly cooked. Give it a stir, and add the remaining cup of broth. Stir until fully incorporated, turn the heat up to high, and cook until the risotto reaches your preferred consistency. I like mine rather thick.
  12. Once the rice is fully cooked, turn off the heat and stir in the grated cheese a bit at a time, reserving some for garnish. Taste, and season with salt and pepper.
  13. Garnish with the grated cheese, and perhaps some herbs. Serve and enjoy!
Variations No mushrooms: Okay, okay, you hate mushrooms! Sorry! You can leave them out. Don't bother with the mushrooms, and add broth to make up for the lost liquid. Mushroom lover: Alternatively, if you love mushrooms and you have fresh ones available, use &frac13 to a pound (150-225g) chopped fresh mushrooms instead of dried. Add them at the same time you add the onions to ensure all the liquid cooks off. Milanese, with stuff: One of my favourite variations on this dish is a version with shrimp, asparagus, and saffron. Leave the mushrooms out, increase the wine to or even 1 cup (seafood likes a little extra acidity), ensure you have a total of 5 cups of broth, and add a few strands of saffron. Chop asparagus and shrimp, season with salt and pepper, saut them with some butter or olive oil in a separate pan so they don't overcook, and fold them in at the very end, after you've added the cheese. I MUST SUFFER: This was too easy? Want to stir more??? Fine, have it your $*!#ing way. I'm sure you can be creative and come up with something on your own...

9 September 2020

Reproducible Builds: Reproducible Builds in August 2020

Welcome to the August 2020 report from the Reproducible Builds project. In our monthly reports, we summarise the things that we have been up to over the past month. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced from the original free software source code to the pre-compiled binaries we install on our systems. If you re interested in contributing to the project, please visit our main website.

This month, Jennifer Helsby launched a new reproduciblewheels.com website to address the lack of reproducibility of Python wheels. To quote Jennifer s accompanying explanatory blog post:
One hiccup we ve encountered in SecureDrop development is that not all Python wheels can be built reproducibly. We ship multiple (Python) projects in Debian packages, with Python dependencies included in those packages as wheels. In order for our Debian packages to be reproducible, we need that wheel build process to also be reproducible
Parallel to this, transparencylog.com was also launched, a service that verifies the contents of URLs against a publicly recorded cryptographic log. It keeps an append-only log of the cryptographic digests of all URLs it has seen. (GitHub repo) On 18th September, Bernhard M. Wiedemann will give a presentation in German, titled Wie reproducible builds Software sicherer machen ( How reproducible builds make software more secure ) at the Internet Security Digital Days 2020 conference.

Reproducible builds at DebConf20 There were a number of talks at the recent online-only DebConf20 conference on the topic of reproducible builds. Holger gave a talk titled Reproducing Bullseye in practice , focusing on independently verifying that the binaries distributed from ftp.debian.org are made from their claimed sources. It also served as a general update on the status of reproducible builds within Debian. The video (145 MB) and slides are available. There were also a number of other talks that involved Reproducible Builds too. For example, the Malayalam language mini-conference had a talk titled , ? ( I want to join Debian, what should I do? ) presented by Praveen Arimbrathodiyil, the Clojure Packaging Team BoF session led by Elana Hashman, as well as Where is Salsa CI right now? that was on the topic of Salsa, the collaborative development server that Debian uses to provide the necessary tools for package maintainers, packaging teams and so on. Jonathan Bustillos (Jathan) also gave a talk in Spanish titled Un camino verificable desde el origen hasta el binario ( A verifiable path from source to binary ). (Video, 88MB)

Development work After many years of development work, the compiler for the Rust programming language now generates reproducible binary code. This generated some general discussion on Reddit on the topic of reproducibility in general. Paul Spooren posted a request for comments to OpenWrt s openwrt-devel mailing list asking for clarification on when to raise the PKG_RELEASE identifier of a package. This is needed in order to successfully perform rebuilds in a reproducible builds context. In openSUSE, Bernhard M. Wiedemann published his monthly Reproducible Builds status update. Chris Lamb provided some comments and pointers on an upstream issue regarding the reproducibility of a Snap / SquashFS archive file. [ ]

Debian Holger Levsen identified that a large number of Debian .buildinfo build certificates have been tainted on the official Debian build servers, as these environments have files underneath the /usr/local/sbin directory [ ]. He also filed against bug for debrebuild after spotting that it can fail to download packages from snapshot.debian.org [ ]. This month, several issues were uncovered (or assisted) due to the efforts of reproducible builds. For instance, Debian bug #968710 was filed by Simon McVittie, which describes a problem with detached debug symbol files (required to generate a traceback) that is unlikely to have been discovered without reproducible builds. In addition, Jelmer Vernooij called attention that the new Debian Janitor tool is using the property of reproducibility (as well as diffoscope when applying archive-wide changes to Debian:
New merge proposals also include a link to the diffoscope diff between a vanilla build and the build with changes. Unfortunately these can be a bit noisy for packages that are not reproducible yet, due to the difference in build environment between the two builds. [ ]
56 reviews of Debian packages were added, 38 were updated and 24 were removed this month adding to our knowledge about identified issues. Specifically, Chris Lamb added and categorised the nondeterministic_version_generated_by_python_param and the lessc_nondeterministic_keys toolchain issues. [ ][ ] Holger Levsen sponsored Lukas Puehringer s upload of the python-securesystemslib pacage, which is a dependency of in-toto, a framework to secure the integrity of software supply chains. [ ] Lastly, Chris Lamb further refined his merge request against the debian-installer component to allow all arguments from sources.list files (such as [check-valid-until=no]) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure and sent a ping to the team that maintains that code.

Upstream patches The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of these patches, including:

diffoscope diffoscope is our in-depth and content-aware diff utility that can not only locate and diagnose reproducibility issues, it provides human-readable diffs of all kinds. In August, Chris Lamb made the following changes to diffoscope, including preparing and uploading versions 155, 156, 157 and 158 to Debian:
  • New features:
    • Support extracting data of PGP signed data. (#214)
    • Try files named .pgp against pgpdump(1) to determine whether they are Pretty Good Privacy (PGP) files. (#211)
    • Support multiple options for all file extension matching. [ ]
  • Bug fixes:
    • Don t raise an exception when we encounter XML files with <!ENTITY> declarations inside the Document Type Definition (DTD), or when a DTD or entity references an external resource. (#212)
    • pgpdump(1) can successfully parse some binary files, so check that the parsed output contains something sensible before accepting it. [ ]
    • Temporarily drop gnumeric from the Debian build-dependencies as it has been removed from the testing distribution. (#968742)
    • Correctly use fallback_recognises to prevent matching .xsb binary XML files.
    • Correct identify signed PGP files as file(1) returns data . (#211)
  • Logging improvements:
    • Emit a message when ppudump version does not match our file header. [ ]
    • Don t use Python s repr(object) output in Calling external command messages. [ ]
    • Include the filename in the not identified by any comparator message. [ ]
  • Codebase improvements:
    • Bump Python requirement from 3.6 to 3.7. Most distributions are either shipping with Python 3.5 or 3.7, so supporting 3.6 is not only somewhat unnecessary but also cumbersome to test locally. [ ]
    • Drop some unused imports [ ], drop an unnecessary dictionary comprehensions [ ] and some unnecessary control flow [ ].
    • Correct typo of output in a comment. [ ]
  • Release process:
    • Move generation of debian/tests/control to an external script. [ ]
    • Add some URLs for the site that will appear on PyPI.org. [ ]
    • Update author and author email in setup.py for PyPI.org and similar. [ ]
  • Testsuite improvements:
    • Update PPU tests for compatibility with Free Pascal versions 3.2.0 or greater. (#968124)
    • Mark that our identification test for .ppu files requires ppudump version 3.2.0 or higher. [ ]
    • Add an assert_diff helper that loads and compares a fixture output. [ ][ ][ ][ ]
  • Misc:
In addition, Mattia Rizzolo documented in setup.py that diffoscope works with Python version 3.8 [ ] and Frazer Clews applied some Pylint suggestions [ ] and removed some deprecated methods [ ].

Website This month, Chris Lamb updated the main Reproducible Builds website and documentation to:
  • Clarify & fix a few entries on the who page [ ][ ] and ensure that images do not get to large on some viewports [ ].
  • Clarify use of a pronoun re. Conservancy. [ ]
  • Use View all our monthly reports over View all monthly reports . [ ]
  • Move a is a suffix out of the link target on the SOURCE_DATE_EPOCH age. [ ]
In addition, Javier Jard n added the freedesktop-sdk project [ ] and Kushal Das added SecureDrop project [ ] to our projects page. Lastly, Michael P hn added internationalisation and translation support with help from Hans-Christoph Steiner [ ].

Testing framework The Reproducible Builds project operate a Jenkins-based testing framework to power tests.reproducible-builds.org. This month, Holger Levsen made the following changes:
  • System health checks:
    • Improve explanation how the status and scores are calculated. [ ][ ]
    • Update and condense view of detected issues. [ ][ ]
    • Query the canonical configuration file to determine whether a job is disabled instead of duplicating/hardcoding this. [ ]
    • Detect several problems when updating the status of reporting-oriented metapackage sets. [ ]
    • Detect when diffoscope is not installable [ ] and failures in DNS resolution [ ].
  • Debian:
    • Update the URL to the Debian security team bug tracker s Git repository. [ ]
    • Reschedule the unstable and bullseye distributions often for the arm64 architecture. [ ]
    • Schedule buster less often for armhf. [ ][ ][ ]
    • Force the build of certain packages in the work-in-progress package rebuilder. [ ][ ]
    • Only update the stretch and buster base build images when necessary. [ ]
  • Other distributions:
    • For F-Droid, trigger jobs by commits, not by a timer. [ ]
    • Disable the Archlinux HTML page generation job as it has never worked. [ ]
    • Disable the alternative OpenWrt rebuilder jobs. [ ]
  • Misc;
Many other changes were made too, including:
  • Chris Lamb:
    • Use <pre> HTML tags when dumping fixed-width debugging data in the self-serve package scheduler. [ ]
  • Mattia Rizzolo:
  • Vagrant Cascadian:
    • Mark that the u-boot Universal Boot Loader should not build architecture independent packages on the arm64 architecture anymore. [ ]
Finally, build node maintenance was performed by Holger Levsen [ ], Mattia Rizzolo [ ][ ] and Vagrant Cascadian [ ][ ][ ][ ]

Mailing list On our mailing list this month, Leo Wandersleb sent a message to the list after he was wondering how to expand his WalletScrutiny.com project (which aims to improve the security of Bitcoin wallets) from Android wallets to also monitor Linux wallets as well:
If you think you know how to spread the word about reproducibility in the context of Bitcoin wallets through WalletScrutiny, your contributions are highly welcome on this PR [ ]
Julien Lepiller posted to the list linking to a blog post by Tavis Ormandy titled You don t need reproducible builds. Morten Linderud (foxboron) responded with a clear rebuttal that Tavis was only considering the narrow use-case of proprietary vendors and closed-source software. He additionally noted that the criticism that reproducible builds cannot prevent against backdoors being deliberately introduced into the upstream source ( bugdoors ) are decidedly (and deliberately) outside the scope of reproducible builds to begin with. Chris Lamb included the Reproducible Builds mailing list in a wider discussion regarding a tentative proposal to include .buildinfo files in .deb packages, adding his remarks regarding requiring a custom tool in order to determine whether generated build artifacts are identical in a reproducible context. [ ] Jonathan Bustillos (Jathan) posted a quick email to the list requesting whether there was a list of To do tasks in Reproducible Builds. Lastly, Chris Lamb responded at length to a query regarding the status of reproducible builds for Debian ISO or installation images. He noted that most of the technical work has been performed but there are at least four issues until they can be generally advertised as such . He pointed that the privacy-oriented Tails operation system, which is based directly on Debian, has had reproducible builds for a number of years now. [ ]
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

5 September 2020

Elana Hashman: Three talks at DebConf 2020

This year has been a really unusual one for in-person events like conferences. I had already planned to take this year off from travel for the most part, attending just a handful of domestic conferences. But the pandemic has thrown those plans into chaos; I do not plan to attend large-scale in-person events until July 2021 at the earliest, per my employer's guidance. I've been really sad to have turned down multiple speaking invitations this year. To try to set expectations, I added a note to my Talks page that indicates I will not be writing any new talks for 2020, but am happy to join panels or reprise old talks. And somehow, with all that background, I still ended up giving three talks at DebConf 2020 this year. In part, I think it's because this is the first DebConf I've been able to attend since 2017, and I was so happy to have the opportunity! I took time off work to give myself enough space to focus on the conference. International travel is very difficult for me, so DebConf is generally challenging if not impossible for me to attend. A panel a day keeps the FTP Team away? On Thursday, August 27th, I spoke on the Leadership in Debian panel, where I discussed some of the challenges leadership in the project must face, including an appropriate response to the BLM movement and sustainability for volunteer positions that require unsustainable hours (such as DPL).
On Friday, August 28th, I hosted the Debian Clojure BoF, attended by members of the Clojure and Puppet teams. The Puppet team is working to package the latest versions of Puppet Server/DB, which involve significant Clojure components, and I am doing my best to help.
On Saturday, August 29th, I spoke on the Meet the Technical Committee panel. The Committee presented a number of proposals for improving how we work within the project. I was responsible for presenting our first proposal on allowing folks to engage the committee privately.

2 September 2020

Elana Hashman: My term at the Open Source Initiative thus far

When I ran for the OSI board in early 2019, I set three goals for myself: Now that the OSI has announced hiring an interim General Manager, I thought it would be a good time to publicly reflect on what I've accomplished and what I'd like to see next. As I promised in my campaign pitch, I aim to be publicly accountable :) Growing the OSI's membership I have served as our Membership Committee Chair since the May 2019 board meeting, tasked with devising and supervising strategy to increase membership and deliver value to members. As part of my election campaign last year, I signed up over 50 new individual members. Since May 2019, we've seen strong 33% growth of individual members, to reach a new all-time high over 600 (638 when I last checked). I see the OSI as a relatively neutral organization that occupies a unique position to build bridges among organizations within the FOSS ecosystem. In order to facilitate this, we need a representative membership, and we need to engage those members and provide forums for cross-pollination. As Membership Committee Chair, I have been running quarterly video calls on Jitsi for our affiliate members, where we can share updates between many global organizations and discuss challenges we all face. But it's not enough just to hold the discussion; we also need to bring fresh new voices into the conversation. Since I've joined the board, I'm thrilled to say that 16 new affiliate members joined (in chronological order) for a total of 81: I was also excited to run a survey of the OSI's individual and affiliate membership to help inform the future of the organization that received 58 long-form responses. The survey has been accepted by the board at our August meeting and should be released publicly soon! Defending the Open Source Definition When I joined the board, the first committee I joined was the License Committee, which is responsible for running the licence review process, making recommendations on new licenses, and maintaining our existing licenses. Over the past year, under Pamela Chestek's leadership as Chair, the full board has approved the following licenses (with SPDX identifiers in brackets) on the recommendation of the License Committee: We withheld approval of the following licenses: I've also worked to define the scope of work for hiring someone to improve our license review process, which we have an open RFP for! Chopping wood and carrying water I joined the OSI with the goal of improving an organization I didn't think was performing up to its potential. Its membership and board were not representative of the wider open source community, its messaging felt outdated, and it seemed to be failing to rise to today's challenges for FOSS. But before one can rise to meet these challenges, you need a strong foundation. The OSI needed the organizational structure, health, and governance in order to address such questions. Completing that work is essential, but not exactly glamourous and it's a place that I thrive. Honestly, I don't (yet?) want to be the public face of the organization, and I apologize to those who've missed me at events like FOSDEM. I want to talk a little about some of my behind-the-scenes activities that I've completed as part of my board service: All of this work is intended to improve the organization's health and provide it with an excellent foundation for its mission. Defining the future of open source Soon after I was elected to the board, I gave a talk at Brooklyn.js entitled "The Future of Open Source." In this presentation, I pondered about the history and future of the free and open source software movement, and the ethical questions we must face. In my election campaign, I wrote "Software licenses are a means, not an end, to open source software. Focusing on licensing is necessary but not sufficient to ensure a vibrant, thriving open source community. Focus on licensing to the exclusion of other serious community concerns is to our collective detriment." My primary goal for my first term on the board was to ensure the OSI would be positioned to answer wider questions about the open source community and its future beyond licenses. Over the past two months, I supported Megan Byrd-Sanicki's suggestion to hold (and then participated in, with the rest of the board) organizational strategy sessions to facilitate our long-term planning. My contribution to help inform these sessions was providing the member survey on behalf of the Membership Committee. Now, I think we are much better equiped to face the hard questions we'll have to tackle. In my opinion, the Open Source Initiative is better positioned than ever to answer them, and I can't wait to see what the future brings. Hope to see you at our first State of the Source conference next week!

28 May 2020

Elana Hashman: Presenter mode in LibreOffice Impress without an external display

I typically use LibreOffice Impress for my talks, much to some folks' surprise. Yes, you can make slides look okay with free software! But there's one annoying caveat that has bothered me for ages. Impress makes it nearly impossible to enter presenter mode with a single display, while also displaying slides. I have never understood this limitation, but it's existed for a minimum of seven years. I've tried all sorts of workarounds over the years, including a macro that forces LibreOffice into presenter mode, which I never was able to figure out how to reverse once I ran it... This has previously been an annoyance but never posed a big problem, because when push came to shove I could leave my house and use an external monitor or screen when presenting at meetups. But now, everything's virtual, I'm in lockdown indefinitely, and I don't have another display available at home. And about 8 hours before speaking at a meetup today, I realized I didn't have a way to share my slides while seeing my speaker notes. Uh oh. So I got this stupid idea. ...why don't I just placate LibreOffice with a FAKE display? Virtual displays with xrandr My GPU had this capability innately, it turns out, if I could just whisper the right incantations to unlock its secrets: 
ehashman@red-dot:~$ cat /usr/share/X11/xorg.conf.d/20-intel.conf 
Section "Device"
    Identifier "intelgpu0"
    Driver "intel"
    Option "VirtualHeads" "1"
EndSection
After restarting X to allow this newly created config to take effect, I now could see two new virtual displays available for use: 
ehashman@red-dot:~$ xrandr
Screen 0: minimum 8 x 8, current 3840 x 1080, maximum 32767 x 32767
eDP1 connected primary 1920x1080+0+0 (normal left inverted right x axis y axis) 310mm x 170mm
   1920x1080     60.01*+  59.93  
   ...
   640x360       59.84    59.32    60.00  
DP1 disconnected (normal left inverted right x axis y axis)
DP2 disconnected (normal left inverted right x axis y axis)
HDMI1 disconnected (normal left inverted right x axis y axis)
HDMI2 disconnected (normal left inverted right x axis y axis)
VIRTUAL1 disconnected (normal left inverted right x axis y axis)
VIRTUAL2 disconnected (normal left inverted right x axis y axis)
Nice. Now, to actually use it: 
ehashman@red-dot:~$ xrandr --addmode VIRTUAL1 1920x1080
ehashman@red-dot:~$ xrandr --output VIRTUAL1 --mode 1920x1080 --right-of eDP1
And indeed, after running these commands, I found myself with a virtual display, very happy to black hole all my windows, available to the imaginary right of my laptop screen. This allowed me to mash that "Present" button in LibreOffice and get my presenter notes on my laptop display, while putting my actual slides in a virtual time-out that I could still screenshare! Wouldn't it be nice if LibreOffice just fixed the actual bug? Well, actually... I must forgive myself for my stage panic. The talk ended up going great, and the immediate problem was solved. But it turns out this bug has been addressed upstream! It's just... not well-documented. A couple years ago, there was a forum post on ask.libreoffice.org that featured this exact question, and a solution was provided!
Yes, use Open Expert Configuration via Tools > Options > LibreOffice > Advanced. Search for StartAlways. You should get a node org.openoffice.Office.PresenterScreen with line Presenter. Double-click that line to toggle the boolean value to true.
I tested this out locally and... yeah that works. But it turns out the bug from 2013 had not been updated with this solution until just a few months ago. There are very limited search results for this configuration property. I wish this was much better documented. But so it goes with free software; here's a hack and a real solution as we all try to improve things :)

25 May 2020

Elana Hashman: Beef and broccoli

Beef and broccoli is one of my favourite easy weeknight meals. It's savoury, it's satisfying, it's mercifully quick, and so, so delicious. The recipe here is based on this one but with a few simplifications and modifications. The ingredients are basically the same, but the technique is a little different. Oh, and I include some fermented black beans because they're yummy :3 and Serves 3. Active time: 20-30m. Total time: 30m. Ingredients: For the marinade: For the sauce: For the rice: Recipe:
  1. Begin by preparing the rice: combine rice, water, and salt in a small saucepan and bring to a boil. Once it reaches a boil, cover and reduce to a simmer for 20 minutes. Then turn off the heat and let it rest.
  2. Mix the marinade in a bowl large enough to hold the meat.
  3. Thinly slice the beef. Place slices in the marinade bowl and toss to evenly coat.
  4. Start heating a wok or similar pan on medium-high to high heat. This will ensure you get a good sear.
  5. Prep the garlic, black beans, and broccoli. If you use frozen broccoli, you can save some time here :)
  6. Combine the ingredients for the sauce and mix well to ensure the corn starch doesn't form any lumps.
  7. By this point the beef should have been marinating for about 10-15m. Add a tablespoon of neutral cooking oil (like canola or soy) to the meat, give it one more toss to ensure it's thoroughly coated, then add a layer of meat to the dry pan. You may need to sear in batches. On high heat, cooking will take 1-2 minutes per side, ensuring each is nicely browned. Once the strips are cooked, remove from the pan and set aside.
  8. While the beef is cooking, steam the broccoli until it's almost (but not quite) cooked through. I typically do this by putting it in a large bowl, adding 2 tbsp water, covering it and microwaving: 3 minutes on high if frozen, 1 minutes on high if fresh, pausing halfway through to stir.
  9. Once all the beef is cooked and set aside, spray the pan or add oil to coat and add the garlic and black bean (if using). Stir and cook for 30-60 seconds until fragrant.
  10. When the garlic is ready, give the sauce a quick stir and add it to the pan, using it to deglaze. Once there are no more bits stuck to the bottom and the sauce is thick to your liking, add the broccoli and beef to the pan. Mix until thoroughly coated in sauce and heated through. Taste and adjust seasoning (salt, pepper, soy, etc.) if necessary.
  11. Fluff the rice and serve. Enjoy!
But I am a vegetarian??? Seitan can substitute for the beef relatively easily. Finding a substitute for oyster sauce is a little bit trickier, especially since it's the star ingredient. You can buy vegetarian or vegan oyster sauce (they're usually mushroom-based), but I have no idea how good they taste. You can also try making it at home! If you do, let me know how it turns out?

9 April 2020

Elana Hashman: Repack Zoom .debs to remove the ibus dependency

For whatever reason, Zoom distributes .debs that have a dependency on ibus. ibus is the "intelligent input bus" package and as far as I'm aware, might be used for emoji input in chat or something?? But is otherwise not actually a dependency of the Zoom package. I've tested this extensively... the client works fine without it. I noticed when I installed ibus along with the Zoom package that ibus would frequently eat an entire core of CPU. I'm sure this is a bug in the ibus package or service, but I have no energy to try to get that fixed. If it's not a hard dependency, Zoom shouldn't depend on it in the first place. Anyways, here's how you can repack a Zoom .deb to remove the ibus dependency: 
scratch=$(mktemp -d)
# Extract package contents
dpkg -x zoom_amd64.deb $scratch
# Extract package control information
dpkg -e zoom_amd64.deb $scratch/DEBIAN
# Remove the ibus dependency
sed -i -E 's/(ibus,  , ibus)//' $scratch/DEBIAN/control
# Rebuild the .deb
dpkg -b $scratch patched_zoom_amd64.deb
Now you can install the patched .deb with 
dpkg -i patched_zoom_amd64.deb
The upstream fix would be for Zoom to move the ibus "Dependency" to a "Recommends", but they have been unwilling to do this for over a year. But wait, what version even is my package? By the way, you may have also noticed that the Zoom client downloads do not conform to the standard Debian package naming scheme (i.e. including the version in the filename). If you're not sure what version a zoom_amd64.deb package you've downloaded is, you can quickly extract that information with dpkg-deb: 
dpkg-deb -I zoom_amd64.deb   grep Version
# Version: 3.5.383291.0407

3 March 2020

Elana Hashman: Chili

I really like beans. This is one way for me to power through the copious amounts I receive through the beloved Bean of the Month Club. (It took me many months, but I finally got in!) Beef Chili Serves 5-7. Active time: 20m. Total time: 90m. Ingredients: Recipe:
  1. Heat a dutch oven on high.
  2. Small or medium dice (depending on your preference) the celery, pepper, and onion. You should have about four cups total.
  3. Brown the beef in the dutch oven, using a little bit of cooking oil to avoid sticking if desired.
  4. While the meat is browning, mince the garlic, measure the spices, and combine these all together.
  5. Once the meat is brown, add the diced vegetables and cook until tender and the onion is translucent.
  6. Add the spices and garlic, thoroughly combine, and cook until fragrant, about one minute.
  7. Add the tomatoes. Once the mixture comes to a boil, cover and simmer for about an hour, stirring every ~20 minutes or so.
  8. Add the beans and use bean broth or water to adjust consistency.
  9. This is a good time to taste and add salt. I usually use 1 to 1 teaspoons.
  10. Bring to a boil again, cover and simmer for 10-15 more minutes.
  11. Turn off the heat, taste, and adjust seasoning if necessary. Add a squeeze of lime if it needs a little acidity.
  12. Serve with sour cream, grated cheese, sliced green onions, or any of your other favourite garnishes.
  13. Enjoy!
Frequently Asked Questions Q: Why wait until basically the end to add salt?
A: Because canned ingredients will contain salt, and it's hard to predict how much, so you should always salt to taste. Waiting until the end will always ensure you season accurately. Q: Why are so many of these directions so hand-wavey?
A: Cooking is an adventure. Experiment a little! Q: But I have no idea what "to taste" means.
A: Taste it, add the thing, stir, taste it again. Is it salty enough? Sour enough? Sweet enough? You don't know, you say? Keep going. Add a little too much and then you'll find out for next time. Q: Why do you add so many beans and so little meat?
A: I really like beans.

2 November 2017

Bits from Debian: New Debian Developers and Maintainers (September and October 2017)

The following contributors got their Debian Developer accounts in the last two months: The following contributors were added as Debian Maintainers in the last two months: Congratulations!

1 June 2017

Markus Koschany: My Free Software Activities in May 2017

Welcome to gambaru.de. Here is my monthly report that covers what I have been doing for Debian. If you re interested in Java, Games and LTS topics, this might be interesting for you. Debian Games Bug fixes New upstream release Debian Java Debian LTS This was my fifteenth month as a paid contributor and I have been paid to work 27,25 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following: Misc Thanks for reading and see you next time.

28 September 2014

Benjamin Mako Hill: Community Data Science Workshops Post-Mortem

Earlier this year, I helped plan and run the Community Data Science Workshops: a series of three (and a half) day-long workshops designed to help people learn basic programming and tools for data science tools in order to ask and answer questions about online communities like Wikipedia and Twitter. You can read our initial announcement for more about the vision. The workshops were organized by myself, Jonathan Morgan from the Wikimedia Foundation, long-time Software Carpentry teacher Tommy Guy, and a group of 15 volunteer mentors who taught project-based afternoon sessions and worked one-on-one with more than 50 participants. With overwhelming interest, we were ultimately constrained by the number of mentors who volunteered. Unfortunately, this meant that we had to turn away most of the people who applied. Although it was not emphasized in recruiting or used as a selection criteria, a majority of the participants were women. The workshops were all free of charge and sponsored by the UW Department of Communication, who provided space, and the eScience Institute, who provided food. cdsw_combo_images-1The curriculum for all four session session is online: The workshops were designed for people with no previous programming experience. Although most our participants were from the University of Washington, we had non-UW participants from as far away as Vancouver, BC. Feedback we collected suggests that the sessions were a huge success, that participants learned enormously, and that the workshops filled a real need in the Seattle community. Between workshops, participants organized meet-ups to practice their programming skills. Most excitingly, just as we based our curriculum for the first session on the Boston Python Workshop s, others have been building off our curriculum. Elana Hashman, who was a mentor at the CDSW, is coordinating a set of Python Workshops for Beginners with a group at the University of Waterloo and with sponsorship from the Python Software Foundation using curriculum based on ours. I also know of two university classes that are tentatively being planned around the curriculum. Because a growing number of groups have been contacting us about running their own events based on the CDSW and because we are currently making plans to run another round of workshops in Seattle late this fall I coordinated with a number of other mentors to go over participant feedback and to put together a long write-up of our reflections in the form of a post-mortem. Although our emphasis is on things we might do differently, we provide a broad range of information that might be useful to people running a CDSW (e.g., our budget). Please let me know if you are planning to run an event so we can coordinate going forward.