Search Results: "Sam Hartman"

28 April 2021

Russell Coker: Links April 2021

Dr Justin Lehmiller s blog post comparing his official (academic style) and real biographies is interesting [1]. Also the rest of his blog is interesting too, he works at the Kinsey Institute so you know he s good. Media Matters has an interesting article on the spread of vaccine misinformation on Instagram [2]. John Goerzen wrote a long post summarising some of the many ways of having a decentralised Internet [3]. One problem he didn t address is how to choose between them, I could spend months of work to setup a fraction of those services. Erasmo Acosta wrote an interesting medium article Could Something as Pedestrian as the Mitochondria Unlock the Mystery of the Great Silence? [4]. I don t know enough about biology to determine how plausible this is. But it is a worry, I hope that humans will meet extra-terrestrial intelligences at some future time. Meredith Haggerty wrote an insightful Medium article about the love vs money aspects of romantic comedies [5]. Changes in viewer demographics would be one factor that makes lead actors in romantic movies significantly less wealthy in recent times. Informative article about ZIP compression and the history of compression in general [6]. Vice has an insightful article about one way of taking over SMS access of phones without affecting voice call or data access [7]. With this method the victom won t notice that they are having their sservice interfered with until it s way too late. They also explain the chain of problems in the US telecommunications industry that led to this. I wonder what s happening in this regard in other parts of the world. The clown code of ethics (8 Commandments) is interesting [8]. Sam Hartman wrote an insightful blog post about the problems with RMS and how to deal with him [9]. Also Sam Whitton has an interesting take on this [10]. Another insightful post is by Selam G about RMS long history of bad behavior and the way universities are run [11]. Cory Doctorow wrote an insightful article for Locus about free markets with a focus on DRM on audio books [12]. We need legislative changes to fix this!

24 March 2021

Sam Hartman: Making our Community Safe: the FSF and rms

I felt disgust and horror when I learned yesterday that rms had returned to the FSF board. When rms resigned back in September of 2019, I was Debian Project Leader. At that time, I felt two things. First, I was happy that the community was finally taking a stand in favor of inclusion, respect, and creating a safe, welcoming place to do our work. It was long past time for rms to move on. But I also felt thankful that rms was not my problem to solve. In significant part because of rms, I had never personally been that involved in the FSF. I considered drafting a statement as Debian Project Leader. I could have talked about how through our Diversity Statement and Code of Conduct we had taken a stand in favor of inclusion and respect. I could have talked about how rms's actions displayed a lack of understanding and empathy and how this created a community that was neither welcoming nor respectful. I didn't. I guess I didn't want to deal with confirming I had sufficient support in the project. I wanted to focus on internal goals, and I was healing and learning from some mistakes I made earlier in the year. It looked like other people were saying what needed to be said and my voice was not required. Silence was a mistake.

It's a mistake I've been making all throughout my interactions with rms. Enough is enough. It's long past time I added my voice to those who cry for accountability and who will not sit aside while rms's disrespect and harm is tolerated.

The first time I was silent about rms was around 15 years ago. I was at a science fiction convention in a crowded party. I didn't know anyone, other than the host of the party. I was out of my depth. I heard his voice---I recognized it from the Share the Software Song. He was hitting on some girl, talking about how he invented Emacs. As best I could tell, she didn't even know what Emacs was. Back then, I wondered what she saw in the interaction; why she stuck around even though she didn't know what he was talking about. I sure didn't want to be around; the interaction between the two of them was making me uncomfortable. Besides, the wings on her costume kept hitting me in the face. So I left as fast as I could.

I've learned a lot about creating safe spaces and avoiding sexual harassment since then. Thinking back, she was probably hitting me because she was trying to back away and getting crowded. If this happened today, I think I would do a better job of owning my responsibility for helping keep the space around me safe. I've learned better techniques for checking in to make sure people around me are comfortable.

I didn't come to silence alone: I had been educated into the culture of avoiding rms and not calling him out. There was a running game in the group of computer security professionals I learned from. The goal was to see how much you could contribute to free software and computer security without being recognized by or interacting with rms. And so, indoctrinated into a culture of silence about the harm that rms caused, I took my first step.

Things weren't much better when I attended Libreplanet 2019 just before taking office as Debian Project Leader. I had stayed away from the conference in large part because of rms. But there were Debian people there, and I was missing community interaction. Unfortunately, I saw that even after the problems of 2018, rms was still treating himself as above community standards. He interrupted speakers, objecting to how they phrased the problem they were considering. After a speech on codes of conduct in the free software community, he cornered the talk organizer to "ask her opinion" about the GNU project's lack of code on conduct. He wasn't asking for an opinion. He was justifying himself; there wasn't much listening in the conversation I heard. Aspects of that conversation crossed professional boundaries for what should be said. The talk organizer was okay--we talked about it after--but if we did a better job of policing our community, that wouldn't even be a question. I think the most telling sign was a discussion with an FSF board member. We were having a great conversation, but he had to interrupt it. He was on rms duty (my words) at the next session. The board had decided it was necessary to have members there so that the staff would not be put in awkward positions by their president. If someone needed to call rms out, it could be a board member rather than the staff members of the conduct team.

And yet again, I held my silence. It's so easy to keep silent. It's not that I never speak up. There are communities where I have called people out. But it's hard to paint that target on yourself. It's hard to engage and to stand strong for a community's standards when you aren't the target. It's hard to approach these problems while maintaining empathy for everyone involved. Some people give into the rage; I don't have that option if I want to be the person I've chosen to be. And so, when I do speak up, the emotional cost is high.

Yet, it's long past time I raised my voice on this issue. Rms has demonstrated that he cannot hold to standards of respect for others, respect for their boundaries, or standards of community safety. We need those standards to be a welcoming community.

If the people who came before me--those who taught me the game of avoiding rms--had spoken up, the community could have healed before I even came on the scene. If I and others had stood up fifteen years ago, we'd have another couple generations who were more used to respect, inclusion, welcoming and safety. The FSF board could have done their job back in 2018. And perhaps if more of us had spoken out in 2019, the FSF board would have found the strength to stand strong and not accept rms's return.

And so, finally, I raise my voice. I signed the open letter calling for the resignation of rms and the entire FSF board. Perhaps if we all get used to raising our voice, it will be easier. Perhaps if we stand together, taking the path of community rather than the path of silence, we'll have the support we need to create communities inclusive enough to welcome everyone who can contribute. For me, I'm done being silent.

There's one criticism of the open letter I'd like to respond to. I've heard concerns about asking for the resignation of the entire FSF board under the understanding that some board members voted against rms's return. It should be obvious why those who voted for rms's return need to resign. But resignation does not always mean you did something wrong. If you find yourself in a leadership role in an organization that takes decisions in significant conflict with your standards of ethics, resignation is also the right path. Staying on the board even if you voted against rms's return means that you consider voting for rms to be a reasonable thing to do. It means that even if you disagreed with it, you can still be part of an organization that takes the path of welcoming rms. At this point, I cannot do that, nor can I support leaders in the FSF who do.

7 September 2020

Arturo Borrero Gonz lez: Debconf 2020 online, summary

Debconf2020 logo Debconf2020 took place when I was on personal vacations time. But anyway I m lucky enough that my company, the Wikimedia Foundation, paid the conference registration fee for me and allowed me to take the time (after my vacations) to watch recordings from the conference. This is my first time attending (or watching) a full-online conference, and I was curious to see first hand how it would develop. I was greatly surprised to see it worked pretty nicely, so kudos to the organization, video team, volunteers, etc! What follows is my summary of the conference, from the different sessions and talks I watched (again, none of them live but recordings). The first thing I saw was the Welcome to Debconf 2020 opening session. It is obvious the video was made with lots of love, I found it entertaining and useful. I love it :-) Then I watched the BoF Can Free Software improve social equality. It was introduced and moderated by Hong Phuc Dang. Several participants, about 10 people, shared their visions on the interaction between open source projects and communities. I m pretty much aware of the interesting social advancement that FLOSS can enable in communities, but sometimes is not so easy, it may also present challenges and barriers. The BoF was joined by many people from the Asia Pacific region, and for me, it has been very interesting to take a step back from the usual western vision of this topic. Anyway, about the session itself, I have the feeling the participants may have spent too much time on presentations, sharing their local stories (which are interesting, don t get me wrong), perhaps leaving little room for actual proposal discussions or the like. Next I watched the Bits from the DPL talk. In the session, Jonathan Carter goes over several topics affecting the project, both internally and externally. It was interesting to know more about the status of the project from a high level perspective, as an organization, including subjects such as money, common project problems, future issues we are anticipating, the social aspect of the project, etc. The Lightning Talks session grabbed my attention. It is usually very funny to watch and not as dense as other talks. I m glad I watched this as it includes some interesting talks, ranging from HAM radios (I love them!), to personal projects to help in certain tasks, and even some general reflections about life. Just when I m writing this very sentence, the video for the Come and meet your Debian Publicity team! talk has been uploaded. This team does an incredible work in keeping project information flowing, and social networks up-to-date and alive. Mind that the work of this team is mostly non-engineering, but still, is a vital part of the project. The folks in session explain what the team does, and they also discuss how new people can contribute, the different challenges related to language barriers, etc. I have to admit I also started watching a couple other sessions that turned out to don t be interesting to me (and therefore I didn t finish the video). Also, I tried to watch a couple more sessions that didn t publish their video recording just yet, for example the When We Virtualize the Whole Internet talk by Sam Hartman. Will check again in a couple of days. It is a real pleasure the video recordings from the conference are made available online. One can join the conference anytime (like I m doing!) and watch the sessions at any pace at any time. The video archive is big, I won t be able to go over all of it. I won t lie, I still have some pending videos to watch from last year Debconf2019 :-)

6 August 2020

Sam Hartman: Good Job Debian: Compatibility back to 1999

So, I needed a container of Debian Slink (2.1), released back in 1999. I expected this was going to be a long and involved process. Things didn't look good from the start:
root@mount-peerless:/usr/lib/python3/dist-packages/sqlalchemy# debootstrap  slink /build/slink2 
http://archive.debian.org/debian                                                               
E: No such script: /usr/share/debootstrap/scripts/slink

Hmm, I thought I remembered slink support for debootstrap--not that slink used debootstrap by default--back when I was looking through the debootstrap sources years ago. Sure enough looking through the changelogs, slink support was dropped back in 2005.
Okay, well, this isn't going to work either, but I guess I could try debootstrapping sarge and from there go back to slink.
Except it worked fine.
Go us!

21 April 2020

Debian Project Leader: Bits from the new DPL

My fellow Debianites, It's been one month, one week and one day since I decided to run for this DPL term. The Debian community has been through a variety of interesting times during the last decade, and instead of focusing on grand, sweeping changes for Debian, core to my DPL campaign was to establish a sense of normality and stability so that we can work on community building, continue to focus on technical excellence and serve our users the best we can. Thing don't always work out as we plan, and for many of us, Debian recently had to take a back seat to personal priorities. Back when I posted my intention to run, there were 125 260 confirmed cases of COVID-19 globally. Today, that number is 20 times higher, with the actual infected number likely to be significantly higher. A large number of us are under lock-down, where we not only fear the disease and its effect on local hospitals and how it will affect our loved ones, but also our very livelihoods and the future of our local businesses and industry. I don't mean to be gloomy with the statement above, I am after all, an optimist - but unfortunately it does get even worse. Governments and corporations around the world have started to take advantage of COVID-19 in negative ways and are making large sweeping changes that undermine the privacy and rights of individuals everywhere. For many reasons, including those above, I believe that the Debian project is more important and relevant now than it's ever been before. The world needs a free, general purpose operating system, unburdened by the needs of profit, which puts the needs of its users first, providing a safe and secure platform for the computing needs of the masses. While we can't control or fix all the problems in the world, we can control our response to it, and be part of the solutions that bring the change we want to see. During my term as DPL, I will be available to help with problems in our community to the maximum extent that my time permits. If we help ourselves, we will be in a better position to help others. If you (or your team) get stuck and are in need of help, then please do not hessitate to e-mail me. A few thank-yous As incoming DPL, I'd like to thank Sam Hartman on behalf of the project for the work that he's done over the last year as DPL. It's a tremendous time commitment that requires constant attention to detail. On Sunday, Sam and I had a handover meeting where we discussed various DPL responsibilities including finances, delegations (including specifics of some delegations), legal matters, outreach and other questions I had. I'd also like to thank Sam for taking the time to do this. Thank you to Sruthi Chadran and Brian Gupta who took the time to also run for DPL this year. Both candidates brought important issues to the forefront and I hope to work with both of them on those in the near future. DPL Blog Today, I've started a new blog for the Debian Project Leader to help facilitate more frequent communication, and to reach a wider audience via Planet Debian. This will contain supplemental information to what I send to the debian-devel-announce mailing list. Want to help? In my platform, I listed some key areas that I'd like to work on. My work won't be limited to those, but it should give you some idea of the type of DPL that I'll be. If you'd like to get involved, feel free to join the #debian-dpl channel on the oftc IRC network, and please introduce yourself along with any areas of interest that you'd like to contribute to.

Debian Project Leader: Bits from the new DPL

My fellow Debianites, It's been one month, one week and one day since I decided to run for this DPL term. The Debian community has been through a variety of interesting times during the last decade, and instead of focusing on grand, sweeping changes for Debian, core to my DPL campaign was to establish a sense of normality and stability so that we can work on community building, continue to focus on technical excellence and serve our users the best we can. Thing don't always work out as we plan, and for many of us, Debian recently had to take a back seat to personal priorities. Back when I posted my intention to run, there were 125 260 confirmed cases of COVID-19 globally. Today, that number is 20 times higher, with the actual infected number likely to be significantly higher. A large number of us are under lock-down, where we not only fear the disease and its effect on local hospitals and how it will affect our loved ones, but also our very livelihoods and the future of our local businesses and industry. I don't mean to be gloomy with the statement above, I am after all, an optimist - but unfortunately it does get even worse. Governments and corporations around the world have started to take advantage of COVID-19 in negative ways and are making large sweeping changes that undermine the privacy and rights of individuals everywhere. For many reasons, including those above, I believe that the Debian project is more important and relevant now than it's ever been before. The world needs a free, general purpose operating system, unburdened by the needs of profit, which puts the needs of its users first, providing a safe and secure platform for the computing needs of the masses. While we can't control or fix all the problems in the world, we can control our response to it, and be part of the solutions that bring the change we want to see. During my term as DPL, I will be available to help with problems in our community to the maximum extent that my time permits. If we help ourselves, we will be in a better position to help others. If you (or your team) get stuck and are in need of help, then please do not hessitate to e-mail me. A few thank-yous As incoming DPL, I'd like to thank Sam Hartman on behalf of the project for the work that he's done over the last year as DPL. It's a tremendous time commitment that requires constant attention to detail. On Sunday, Sam and I had a handover meeting where we discussed various DPL responsibilities including finances, delegations (including specifics of some delegations), legal matters, outreach and other questions I had. I'd also like to thank Sam for taking the time to do this. Thank you to Sruthi Chadran and Brian Gupta who took the time to also run for DPL this year. Both candidates brought important issues to the forefront and I hope to work with both of them on those in the near future. DPL Blog Today, I've started a new blog for the Debian Project Leader to help facilitate more frequent communication, and to reach a wider audience via Planet Debian. This will contain supplemental information to what I send to the debian-devel-announce mailing list. Want to help? In my platform, I listed some key areas that I'd like to work on. My work won't be limited to those, but it should give you some idea of the type of DPL that I'll be. If you'd like to get involved, feel free to join the #debian-dpl channel on the oftc IRC network, and please introduce yourself along with any areas of interest that you'd like to contribute to.

19 April 2020

Bits from Debian: DPL elections 2020, congratulations Jonathan Carter!

The Debian Project Leader elections just finished and the winner is Jonathan Carter! His term as project leader starts next Tuesday April 21st and expires on April 20th 2021. Of a total of 1011 developers, 339 developers voted using the Condorcet method. More information about the result is available in the Debian Project Leader Elections 2020 page. Many thanks to Jonathan Carter, Sruthi Chandran and Brian Gupta for running. And special thanks to Sam Hartman for his service as DPL during these last twelve months!

9 March 2020

Sam Hartman: Forged Email

Last night, a series of forged emails was sent to a number of places around the Debian, Ubuntu and Free Software communities. The meat of the mail was a fake message from me to debian-private with the subject "DebConf19 Diversity Girls." I didn't write such a message.
I view this message as the latest installment in a campaign of attacks on Debian that attempt to undermine the project and take up the time of our members.
I was expecting something like this: yesterday, I banned Daniel Pocock from the project.There's been a pattern of related events over the past year and a half:

27 October 2017

Sam Hartman: How Can Debian Turn Disagreement into Something that Makes us Stronger

Recently, when asked to engage with the Debian Technical Committee, a maintainer chose to orphan their package rather than discuss the issue brought before the committee. In another decision earlier this year, a maintainer orphaned their package indicating a lack of respect for the approach being taken and the process. Unfortunately, this joins an ever longer set of issues where people walk away from the TC process disheartened and upset.

For me personally the situations where maintainers walked away from the process were hard. People I respect and admire were telling me that they were unwilling to participate in our dispute resolution process. In one case the maintainer explicitly did not respect a process I had been heavily involved in. As someone who values understanding and build a team, I feel disappointed and hurt thinking about this.

Unfortunately, I don't feel much better when I consider several of the other issues brought before the TC. In a number of cases, the process has concluded, but it feels like a pyrrhic victory. We have an answer, but often we never reached an understanding of one of the key stakeholder's positions. People are sufficiently disappointed or frustrated that they reduce their involvement. That is, the answer is not one they can live with. Sometimes, I'm not even sure that answering the question was worth the cost.

My initial suspicion as I begin to consider issues that have come before the TC in the last few years is that as a necessary consequence of how the TC operates, we will drive a significant chunk of those who come to us away from our community. I will not be part of that. If I end up eventually agreeing with this suspicion, I will either work to improve the TC process or find a different way to contribute to the project that is more aligned with my goals.


Our Community is More Important than Technical Correctness I firmly believe that Debian's strength is its community. Distributions, upstreams, free-software advocates, corporate interests, and everything in between work together. Because we are working on a concrete product,we actually have to understand how our needs conflict and compliment each other. Also, like any organization, our ability to get things done is dependent on our contributors and the time we all put in.

I cannot think of anything more harmful we can do than drive away a constructive contributor. Technical quality is important: many of us will eventually go away if quality drops too much. However, Debian itself can be improved incrementally. Yes, new people do join Debian. However, it takes a lot of new people to make up for a situation where everyone involved is frustrated, and some leave and tell their friends to stay away from Debian.

When a Disagreement Reaches the TC When a disagreement reaches the TC, we know a few things. First, people have not have not been able to resolve it on their own. Second, the disagreement is important enough to at least one of the parties to ask for outside help.

If the TC were to simply announce a decision, it is very likely that at least one party would feel frustrated and disappointed. If the decision is important to someone it almost always means that they care about the outcome. If previous efforts have not reached agreement then there is an outcome that is undesired. While it's possible under the constitution that two parties could bring an issue to the TC mutually where the dynamics could be different, this does not happen in practice.

When we "lose" When a decision making process decides to select one of my undesired outcomes, I have strong negative feelings. First, there is the technical result itself. Because of an adverse decision it is generally harder for me to do my technical work, or some ethical position or group I care about is disadvantaged. However, the strongest feelings are related to needs about my place in the community, not a direct response to the technical decision. I worry about whether issues I care about will be valued in the future; I would likely feel weary or scared thinking of contributing in an environment where issues that matter to me aren't going to be considered. I tend to feel frustrated if my position was not adequately considered this time around.

Often, the strongest feelings do not stem from how I am impacted by the decision itself. Thus, if my more general needs are addressed, I will feel a lot better about not having my preferred outcome selected. Confidence that my position was understood is the single biggest factor in being able to accept the result. If the community took the time to understand me, then I have confidence that I am valued. I can advocate for change and be part of the ongoing discussion. If I understand the other positions then I can understand that the position was not arbitrary. Understanding the other positions is also a prerequisite to seeing how things I value were considered, especially when the tradeoff did not ultimately favor these values.

My conversations with others about their experience with conflict suggest my feelings and needs are common.


However, the TC focuses almost exclusively on the technical matter before it. I don't think the TC has done a good job of actually understanding maintainers or the people bringing issues to us. Especially when we're fairly sure we understand what the ultimate decision will be, we focus on getting to that decision. Of course part of actually understanding someone is considering what they have to say. Even when you have high confidence in an outcome, if you want someone to feel understood, you do have to listen at a point where you can consider what they bring forward.

Frustration of Being Questioned

On the other side, it's frustrating to have your decisions questioned. Reviewing a decision takes time. Even if the TC agrees with a maintainer, asking that maintainer to sit through a long process can create frustrations as deep as any I discussed in the previous section.

So the process is doubly bad for maintainers. Someone can bring up an issue which requires precious time. Then if the TC decides against the maintainer, we have all the problems I discussed previously.

I acknowledge this. A good process must respect the maintainer's time. However, it must respect the community members who disagree with maintainers. Everyone who brings an issue to the TC is a developer. They have contributed significantly to our community and are part of our team. Yes, we need to protect the process against abuse. But in a very real sense if we aren't willing to consider an issue and we aren't willing to engage with someone, understand why they think the issue should be considered, and work until they understand why we made our decision, we are saying we do not respect them enough to take that time. We should expect that to drive people away.

I wonder if the solution here involves a two phase process. During the first phase we work with someone bringing an issue to us until we understand it. We either engage the maintainer at that point, spend the time to help the person bringing an issue to us understand why we are not engaging the maintainer, or have decided the issue is abusive of the processs/misdirected. For this to work maintainers would have to trust us enough to actually be willing to sit back and not spend a lot of their time.

Sam, It's Just Personalities

One criticism I've heard as I discuss this is that a lot of the negative feelings surround interpersonal conflict and are personality conflicts. Yes, personalities matter. Yes, we re still healing from the Systemd discussion.

However, as a community, I think we need to figure out how we respect the inevitable personality conflicts. I can think of one or two developers I'm still upset with years after an issue. It happens. Perhaps if I were a maintainer when one of these developers raised a concern with my packages, I could ask that someone else be a primary advocate for an issue. Similarly, if a developer wanted to address an issue with me as a maintainer but would prefer not to deal with me, perhaps we could figure out some way that we could see if someone else shared the concern.

Dropping a Package is Sometimes an Answer

My concerns were sparked by instances where a maintainer dropped a package rather than interact with the TC. Sometimes that can be a healthy step in a transition. At Debconf this year, Enrico Zini gave a talk on consent culture in Debian. One of the key points of his talk is that we can find over time that what we're being asked to do in Debian is no longer consistent with our boundaries. If it isn t helping us move forward if it isn t fun then it is likely time to stop.

In that sense, approaching disagreement can be a great time to take stock and ask whether we still enjoy maintaining a package. If we don t, then stepping aside and letting others take it on can be a great way that we and they can be happier. I don t really think that s what happened in these instances though. Based on comments made to me, this sounded more like a lack of faith in being treated well or in our ability as a committee.

Even so, Enrico s talk applies in a number of ways here. He suggests that the approach we should take when someone is done and steps away is to thank then. I couldn t agree more. From the depth of my heart I offer thanks: Thank you for taking care of yourself and stepping away when it is no longer fun. Thank you for all the effort you have put into these packages. I hope to work with you on great things in the future.

12 August 2017

Sam Hartman: Debian: a Commons of Innovation

I recently returned from Debconf. This year at Debconf, Matthew Garrett gave a talk about the next twenty years in free software. In his talk he raised concerns that Debian might not be relevant in that ecosystem and talked about some of the trends that contribute to his concerns.
I was talking to Marga after the talk and she said that Debian used to be a lot more innovative than it is today.
My initial reaction was doubt; what she said didn't feel right to me. At the time I didn't have a good answer. Since then I've been pondering the issue, and I think I have a partial answer to both Marga and Matthew and so I'll share it here.
In the beginning Debian focused on a lot of technical innovations related to bringing an operating system together. We didn't understand how to approach builds and build dependencies in a uniform manner. Producing packages in a clean environment was hard. We didn't understand what we wanted out of packages in terms of a uniform approach to configuration handling and upgrades. To a large extent we've solved those problems.
However, as the community has grown, our interests are more diverse. Our users and free software (and the operating system we build together) are what bring us together: we still have a central focus. However, no one technical project captures us all. There's still significant technical innovation in the Debian ecosystem. That innovation happens in Debian teams, companies and organizations that interact with the Debian community. We saw several talks about such innovation this year. I found the talk about ostree and flatpak interesting, especially because it focused on people in the broader Debian ecosystem valuing Debian along with some of the same technologies that Matthew is worried will undermine our relevance.
Matthew talked about how Debian ends up being a man-in-the-middle. We're between users and developers. we're between distributions and upstreams. Users are frustrated because we hold back the latest version of software they want from getting to them. Developers are frustrated because we present our users with old versions of their software configured not as they would like, combined with different dependencies than they expect.
All these weaknesses are real.
However, I think Debian-in-the-middle is our greatest strength both on the technical and social front.
I value Debian because I get a relatively uniform interface to the software I use. I can take one approach to reporting bugs whether they are upstream or Debian specific. I expect the software to behave in uniform ways with regard to things covered by policy. I know that I'm not going to have to configure multiple different versions of core dependencies; for the most part system services are shared. When Debian has value it's because our users want those things we provide. Debian has also reviewed every source file in the software we ship to understand the license and license compatibility. As a free software supporter and as someone who consumes software in commercial context, that value alone is enormous.
The world has evolved and we're facing technologies that provide different models. They've been coming for years: Python, Ruby, Java, Perl and others have been putting together their own commons of software. They have all been working to provide virtualization to isolate one program (and its dependencies) from another. Containerization takes that to the next level. Sometimes that's what our users want.
We haven't figured out what the balances are, how we fit into this new world. However, I disagree with the claim that we aren't even discussing the problem. I think we're trying a lot of things off in our own little technical groups. We're just getting to the level of critical mass of understanding where we can take advantage of Debian's modern form of innovation.
Because here's the thing. Debian's innovation now is social, not technical. Just as we're in the middle technically, we're in the middle socially. Upstreams, developers, users, derivatives, and all the other members of our community work together. we're a place where we can share technology, explore solutions, and pull apart common elements. This is the first Debconf where it felt like we'd explored some of these trends enough to start understanding how they might fit together in a whole. Seven years ago, it felt like we were busy being convinced the Java folks were wrong-headed. A couple of years later, it felt like we were starting to understand our users' desires that let to models different than packaging, but we didn't have any thoughts. At least in my part of the hallway it sounded like people were starting to think about how they might fit parts together and what the tradeoffs would be.
Yes, Matthew's talk doubtless sparked some of that. I think he gave us a well deserved and important wake-up call. However, I was excited by the discussion prior to Thursday.
What I'm taking a way is that Debian is valuable when there's a role in the middle. Both socially and technically we should capitalize on situations where something between makes things better and get out of the way where it does not.

9 April 2017

Sam Hartman: When "when" is too hard a question: SQLAlchemy, Python datetime, and ISO8601

A new programmer asked on a work chat room how timezones are handled in databases. He asked if it was a good idea to store things in UTC. The senior programmers all laughed as we told some of our horror stories with timezones. Yes, UTC is great; if only it were that simple.
About a week later I was designing the schema for a blue sky project I'm implementing. I had to confront time in all its Pythonic horror.
Let's start with the datetime.datetime class. Datetime objects optionally include a timezone. If no timezone is present, several methods such as timestamp treat the object as a local time in the system's timezone. The timezone method returns a POSIX timestamp, which is always expressed in UTC, so knowing the input timezone is important. The now method constructs such an object from the current time.
However other methods act differently. The utcnow method constructs a datetime object that has the UTC time, but is not marked with a timezone. So, for example datetime.fromtimestamp(datetime.utcnow().timestamp()) produces the wrong result unless your system timezone happens to have the same offset as UTC.
It's also possible to construct a datetime object that includes a UTC time and is marked as having a UTC time. The utcnow method never does this, but you can pass the UTC timezone into the now method and get that effect. As you'd expect, the timestamp method returns the correct result on such a datetime.
Now enter SQLAlchemy, one of the more popular Python ORMs. Its DATETIME type has an argument that tries to request a column capable of storing a a timezone from the underlying database. You aren't guaranteed to get this though; some databases don't provide that functionality. With PostgreSQL, I do get such a column, although something in SQLAlchemy is not preserving the timezones (although it is correctly adjusting the time). That is, I'll store a UTC time in an object, flush it to my session, and then read back the same time represented in my local timezone (marked as my local timezone). You'd think this would be safe.
Enter SQLite. SQLite makes life hard for people wanting to store time; it seems to want to store things as strings. That's fairly incompatible with storing a timezone and doing any sort of comparisons on dates. SQLAlchemy does not try to store a timezone in SQLite. It just trims any timezone information from the datetime. So, if I do something like
d = datetime.now(timezone.utc)
obj.date_col = d
session.add(obj)
session.flush()
assert obj.date_col == d # fails
assert obj.date_col.timestamp() == d.timestamp() # fails
assert d == obj.date_col.replace(tzinfo = timezone.utc) # finally succeeds

There are some unfortunate consequences of this. If you mark your datetimes with timezone information (even if it is always the same timezone), whether two datetimes representing the same datetime compare equal depends on whether objects have been flushed to the session yet. If you don't mark your objects with timezones, then you may not store timezone information on other databases.
At least if you use only the methods we've discussed so far, you're reasonably safe if you use local time everywhere in your application and don't mark your datetimes with timezones. That's undesirable because as our new programmer correctly surmised, you really should be using UTC. This is particularly true if users of your database might span multiple timezones.
You can use UTC time and not mark your objects as UTC. This will give the wrong data with a database that actually does support timezones, but will sort of work with SQLite. You need to be careful never to convert your datetime objects into POSIX time as you'll get the wrong result.
It turns out that my life was even more complicated because parts of my project serialize data into JSON. For that serialization, I've chosen ISO 8601. You've probably seen that format: '2017-04-09T18:17:27.340410+00:00. Datetime provides the convenient isoformat method to print timestamps in the ISO 8601 format. If the datetime has a timezone indication, it is included in the ISO formatted string. If not, then no timezone indication is included. You know how I mentioned that datetime takes a string without a timezone marker as local time? Yeah, well, that's not what 8601 does: UTC all the way, baby! And at least the parser in the iso8601 module will always include timezone markers. So, if you use datetime to print a timestamp without a timezone marker and then read that back in to construct a new datetime on the deserialization side, then you'll get the wrong time. OK, so mark things with timezones then. Well, if you use local time, then the time you get depends on whether you print the ISO string before or after session flush (before or after SQLAlchemy trims the timezone information as it goes to SQLite).
It turns out that I had the additional complication of one side of my application using SQLite and one side using PostgreSQL. Remember how I mentioned that something between SQLAlchemy and PostgreSQL was recasting my times in local timezone (although keeping the time the same)? Well, consider how that's going to work. I serialize with the timezone marker on the PostgreSQL side. I get a ISO8601 localtime marked with the correct timezone marker. I deserialize on the SQLite side. Before session flush, I get a local time marked as localtime. After session flush, I get a local time with no marking. That's bad. If I further serialize on the SQLite side, I'll get that local time incorrectly marked as UTC. Moreover, all the times being locally generated on the SQLite side are UTC, and as we explored, SQLite really only wants one timezone in play.
I eventually came up with the following approach:

  1. If I find myself manipulating a time without a timezone marking, assert that its timezone is UTC not localtime.

  2. Always use UTC for times coming into the system.

  3. If I'm generating an ISO 8601 time from a datetime that has a timezone marker in a timezone other than UTC, represent that time as a UTC-marked datetime adjusting the time for the change in timezone.


This is way too complicated. I think that both datetime and SQLAlchemy's SQLite time handling have a lot to answer for. I think SQLAlchemy's core time handling may also have some to answer for, but I'm less sure of that.

29 January 2017

Sam Hartman: Network Audio Visualization: Network Modeling

Previously, I wrote about my project to create an audio depiction of network traffic. In this second post, I explore how I model aspects of the network that will be captured in the audio representation. Before getting started, I'll pass along a link. This is not the first time someone has tried to put sound to packets flying through the ether: I was pointed at Peep. I haven't looked at Peep, but will do so after I finish my own write up. Not being an academic, I feel no obligation to compare and contrast my work to others:-)
I started with an idea of what I'd like to hear. One of my motivations was to explore some automated updates we run at work. So, I was hoping to capture the initial DNS and ARP traffic as the update discovered the systems it would contact. Then I was hoping to capture the ssh and other traffic of the actual update.
To Packet or Stream
One of the simplest things to do would simply be to model network packets. For DNS I chose that approach. I was dubious that a packet-based model would capture the aspects of TCP streams I typically care about. I care about the source and destination (both address and port) of course. However I also care about how much traffic is being carried over the stream and the condition of the stream. Are there retransmits? Are there a bunch of unanswered SYNs? But I don't care about the actual distribution of packets. Also, a busy TCP stream can generate thousands of packets a second. I doubted my ability to distinguish thousands of sounds a second at all, especially while trying to convey enough information to carry stream characteristics like overall traffic volume.
So, for TCP, I decided to model some characteristics of streams rather than individual packets.
For DNS, I decided to represent individual requests/replies.
I came up with something clever for ARPP. There, I model the request/reply as an outstanding request. A lot of unanswered ARPs can be a sign of a scan or a significant problem. The mornful sound of a TCP stream trailing off into an unanswered ARP as the cache times out on a broken network is certainly something I'd like to capture. So, I track when an ARP request is sent and when/if it is answered.
Sound or Music
I saw two approaches. First, I could use some sound to represent streams. As an example, a running diesel engine could make a great representation of a stream. The engine speed could represent overall traffic flow. There are many opportunities for detuning the engine to represent various problems that can happen with a stream. Perhaps using stereo separation and slightly different fundamental frequencies I could even represent a couple of streams and still be able to track them.
However, at least with me as a listener, that's not going to scale to a busy network. The other option I saw was to try and create melodic music with various musical phrases modified as conditions within the stream or network changed. That seemed a lot harder to do, but humans are good at listening to complicated music.
I ended up deciding that at least for the TCP streams, I was going to try and produce something more musical than sound. I was nervous: I kept having visions of a performance of "Peter and the Wolf" with different instruments representing all the characters that somehow went dreadfully wrong.
As an aside, the decision to approach music rather than sound depended heavily on what I was trying to capture. If I'm modeling more holistic properties of a system--for example, total network traffic without splitting into streams--I think parameterized sounds would be a better approach.
The decision to approach things musically affected the rest of the modeling. Somehow I was going to need to figure out notes to play. I'd already rejected the idea of modeling packets, so I wouldn't simply be able to play notes when a packet arrived.
Energy Decay
As I played with various options, I realized that the critical challenge would be figuring out how to focus the listener's attention on the important aspects of what was going on. Clutter was the great enemy. My job would be figuring out how to spend sound wisely. When something interesting happened, that part of the model should get more focus--more of the listener's energy.
Soon I found myself thinking a lot about managing the energy of network streams. I imagined streams getting energy when something happened, and spending that energy to convey that interesting event to the listener. Energy needed to accumulate fast enough that even low-traffic streams could be noticed. Energy needed to be spent fast enough that old events were not taking listener focus from new, interesting things going on. However, if the energy were spent slow enough, then network events could be smoothed out to give a better picture of the stream rather than individual packets.
This concept of managing some decaying quantity and managing the rate of decay proved useful at multiple levels of the model.
Two Layer Model
I started with a python script that parses tcpdump output. It associates a packet with a stream and batches packets together to avoid overloading other parts of the system.
The output of this script are stream events. Events include a source and destination address, a stream ID, traffic in each direction, and any special events on the stream.
For DNS, the script just outputs packet events. For ARP, the script outputs request start, reply, and timeout events. There's some initial support for UDP, but so far that doesn't make sound.
Right now, FINs are modeled, but SYNs and the interesting TCP conditions aren't directly modeled. If you get retransmissions you'll notice because packet flow will decrease. However, I'd love to explicitly sound retransmissions. I also think a window filling as an application fails to read is important. I imagine either narrowing a band-pass filter to clamp the audio bandwidth available to a stream with a full window. Or perhaps taking it the other direction and adding an echo.
The next layer down tracks the energy of each stream. But that, and how I map energy into music, is the topic of the next post.

23 January 2017

Sam Hartman: Cudos to GDB Contributors

I recently diagnosed a problem in Debian's pam-p11 package. This package allegedly permits logging into a computer using a smart card or USB security token containing an ssh key. If you know the PIN and have the token, then your login attempt is authorized against the ssh authorized keys file. This seems like a great way to permit console logins as root to machines without having a shared password. Unfortunately, the package didn't work very well for me. It worked once, then all future attempts to use it segfaulted. I'm familiar with how PAM works. I understand the basic ideas behind PKCS11 (the API used for this type of smart card), but was completely unfamiliar with this particular PAM module and the PKCS11 library it used. The segfault was in an area of code I didn't even expect that this PAM module would ever call. Back in 1994, that would have been a painful slog. Gdb has improved significantly since then, and I'd really like to thank all the people over the years who made that possible. I was able to isolate the problem in just a couple of hours of debugging. Here are some of the cool features I used: Anyway, with just a couple of hours and no instrumentation of the code, I managed to track down how a bunch of structures were being freed as an unexpected side effect of one of the function calls. Neither I nor the author of the pam-p11 module expected that (although it is documented and does make sense in retrospect). Good tools make life easier.

15 January 2017

Sam Hartman: Musical Visualization of Network Traffic

I've been working on a fun holiday project in my spare time lately. It all started innocently enough. The office construction was nearing its end, and it was time for my workspace to be set up. Our deployment wizard and I were discussing. Normally we stick two high-end monitors on a desk. I'm blind; that seemed silly. He wanted to do something similarly nice for me, so he replaced one of the monitors with excellent speakers. They are a joy to listen to, but I felt like I should actually do something with them. So, I wanted to play around with some sort of audio project.
I decided to take a crack at an audio representation of network traffic. The solaris version of ping used to have an audio option, which would produce sound for successful pings. In the past I've used audio queues to monitor events like service health and build status.
It seemed like you could produce audio to give an overall feel for what was happening on the network. I was imagining a quick listen would be able to answer questions like:

  1. How busy is the network

  2. How many sources are active

  3. Is the traffic a lot of streams or just a few?

  4. Are there any interesting events such as packet loss or congestion collapse going on?

  5. What's the mix of services involved


I divided the project into three segments, which I will write about in future entries:

I'm fairly happy with what I have. It doesn't represent all the items above. As an example, it doesn't directly track packet loss or retransmissions, nor does it directly distinguish one service from another. Still, just because of the traffic flow, rsync sounds different from http. It models enough of what I'm looking for that I find it to be a useful tool. And I learned a lot about music and Linux audio. I also got to practice designing discrete-time control functions in ways that brought back the halls of MIT.

1 January 2017

Sam Hartman: 2016

I was in such a different place at the beginning of 2016: I was poised to continue to work to help the world find love. Professionally, I was ready to make a much needed transition and find new projects to work on. The year 2016 sucked. It feels like the year was filled with many different versions of the universe saying "Not interested in what you have to offer." At the beginning of the year, I had the energy to try and reach across large disagreements and help find common ground even when compromise was not possible. Now, my blog lies fallow because I cannot find the strength to be vulnerable enough to write what I would choose to say. Certainly a lot of the global changes of the last year have felt like a strong rejection of the world I'd like to see. However, many of the rejections have been personal. Beyond that, most of the people who stood as pillars of support in my life, together helping me find the strength to be vulnerable, are no longer available. When the universe sends such strong messages, it's a good idea to ask whether you are on the right path. I certainly have discovered training I need and things I need to improve in order to avoid making costly mistakes that hurt others. However, among the rejections were clear demonstrations of the value of reaching out with love and compassion. Besides, this is what I'm called to do. It's what I want to do. I certainly will not force it on anyone. But it looks like the next few years may be a hard struggle to find pockets of people interested in that work, finding people who will choose love even in the current world, along with some difficult training to learn from challenges of the last year. Amongst all this, my life if filled with love. There are new connections even as old connections are strained. There is always the hope of finding new ways to connect when the old ones are no longer right. I will rebuild and regain safety. I have the tools to do that. The process is just long and complicated.

11 September 2016

Niels Thykier: Unseen changes to lintian.d.o

We have been making a lot of minor changes to lintian.d.o and the underlying report framework. Most of them were hardly noticeable to the naked. In fact, I probably would not have spotted any of them, if I had not been involved in writing them. Nonetheless, I felt like sharing them, so here goes. User visible changes: In case you were wondering, the section title is partly a pun as half of these changes were intended to assist visually impaired users. They were triggered by me running into Sam Hartmann at DebConf16, where I asked him about how easy Debian s websites were for blind people. Allegedly, we are generally doing quite good in his opinion (with one exception, for which Sam filed Bug#830213), which was a positive surprise for me. On a related note: Thanks Luke Faraone and Asheesh Laroia for getting helping me started on these changes. Reporting framework / Internal changes: With the last change + the no generate reports option, we were able to schedule lintian more frequently. Originally, lintian only ran once a day. With the no generate reports , we added a second run and with the last changes, we bumped it to 4 times a day. Unsurprisingly, it means that we are now reprocessing the archive a lot faster than previously. All of the above is basically the all the note-worthy changes on the Lintian reporting framework since the Partial rewrite of lintian s reporting setup (~1 years ago).
Filed under: Debian, Lintian

10 November 2014

John Goerzen: Debian A plea to worry about what matters, and not take ourselves too seriously

I posted this on debian-devel today. I am also posting it here, because I believe it is important to more than just Debian developers. Good afternoon, This message comes on the heels of Sam Hartman s wonderful plea for compassion [1] and the sad news of Joey Hess s resignation from Debian [2]. I no longer frequently post to this list, but when you ve been a Debian developer for 18 years, and still care deeply about the community and the project, perhaps you have a bit of perspective to share. Let me start with this: Debian is not a Free Software project. Debian is a making-the-world-better project, a caring for people project, a freedom-spreading project. Free Software is our tool. As many of you, hopefully all of you, I joined Debian because I enjoyed working on this project. We all did, didn t we? We joined Debian because it was fun, because we were passionate about it, because we wanted to make the world a better place and have fun doing it. In short, Debian is life-giving, both to its developers and its users. As volunteers, it is healthy to step back every so often, and ask ourselves two questions: 1) Is this activity still life-giving for me? 2) Is it life-giving for others? I have my opinions about init. Strong ones, in fact. [3] They re not terribly relevant to this post. Because I can see that they are not really all that relevant. 14 years ago, I proposed what was, until now anyhow, one of the most controversial GRs in Debian history. It didn t go the way I hoped. I cared about it deeply then, and still care about the principles. I had two choices: I could be angry and let that process ruin my enjoyment of Debian. Or I could let it pass, and continue to have fun working on a project that I love. I am glad I chose the latter. Remember, for today, one way or another, jessie will still boot. 18 years ago when I joined Debian, our major concerns were helping newbies figure out how to compile their kernels, finding manuals for monitors so we could set the X modelines properly, finding some sort of Free web browser, finding some acceptable Office-type software. Wow. We WON, didn t we? Not just Debian, but everyone. Freedom won. I promise you 18 years from now, it will not matter what init Debian chose in 2014. It will probably barely matter in 3 years. This is not key to our goals of making the world a better place. Jessie will still boot. I say that even though my system runs out of memory every few days because systemd-logind has a mysterious bug [4]. It will be fixed. I say that even though I don t know what init system it will use, or how much choice there will be. I say that because it is simply true. We are Debian. We will make it work, one way or another. I don t post much on this list anymore because my personal passion isn t with posting on this list anymore. I make liberal use of my Delete Thread keybinding on -vote these days, because although I care about the GR, I don t care about it enough to read all the messages about it. I have not yet decided if I will spend the time researching it in order to vote. Instead of debating the init GR, sometimes I sit on the sofa with my wife. Sometimes I go out and fly the remote-control airplane I m learning to fly. Sometimes I repair my plane after a flight that was shorter than planned. Sometimes I play games with my boys, or help them with homework, or share my 8-year-old s delight as a text file full of facts about the Titanic that he wrote in Emacs comes spitting out of the printer. Sometimes I write code or play with the latest Linux filesystems or build a new server for my basement. All these things matter more to me than init. I have been using Debian at home for almost 20 years, at various workplaces for almost that long, and it is not going to stop being a part of my life any time soon. Perhaps I will have to learn how to administer a new init system. Well, so be it; I enjoy learning new things. Or perhaps I will have to learn to live with some desktop limitations with an old init system. Well, so be it; it won t bother me much anyhow. Either way, I m still going to be using what is, to me, the best operating system in the world, made by one of the world s foremost Freedom projects. My hope is that all of you may also have the sense of peace I do, that you may have your strong convictions, but may put them all in perspective. That we as a project realize that the enemy isn t the lovers of the other init, but the people that would use law and technology to repress people all over the world. We are but one shining beacon on a hill, but the world will be worse off if our beacon winked out. My plea is that we each may get angry at what matters, and let go of the smaller frustrations in life; that we may each find something more important than init/systemd to derive enjoyment and meaning from. [5] May you each find that airplane to soar freely in the skies, to lift your soul so that the joy of using Free Software to make the world a better place may still be here, regardless of what /sbin/init is. [1] https://lists.debian.org/debian-project/2014/11/msg00002.html [2] https://lists.debian.org/debian-devel/2014/11/msg00174.html [3] A hint might be that in my more grumpy moments, I realize I haven t ever quite figured out why the heck this dbus thing is on so many of my systems, or why I have to edit XML to configure it ;-) [4] #765870 [5] No disrespect meant to the init/systemd maintainers. Keep enjoying what you do, too!

8 February 2014

Sam Hartman: IETF Takes on Challenge of Internet Monitoring

At IETF 88, we held a plenary discussion of how we could harden the Internet against ongoing monitoring and survalence. There were no significant surprises in what people said about monitoring. So, we had an opportunity to focus on what the IETF as a standards organization responsible for technical standards that drive the Internet can do about this problem. I think we made amazing progress. The IETF works by consensus. We discuss issues, and see if there s some position that can gain a rough consensus of the participants in the discussion. After a couple of hours of discussion, Russ Housley asked several consensus questions. The sense of the room is that we should treat these incidents of monitoring as an attack and include them in the threats we try and counter with security features in our protocols. The room believes that encryption, even without authentication has value in fighting these attacks. There was support for the idea of end-to-end encryption is valuable even when there are middle boxes. IETF decisions made in meetings are confirmed on public mailing lists, so the sense of the room is not final. Also, note that I did not capture the exact wording of the questions that were posed. This is huge. There is very strong organizational agreement that we re going to take work in this space as seriously. Now that we ve decided pervasive monitoring is an attack, anyone can ask how a proposed protocol (or change to a protocol) counters that attack. If it doesn t handle the attack and there is a way to address the attack, then we will be in a stronger position arguing the threat could be addressed. In addition, the commitment to encryption providing value without authentication will be useful in providing privacy and minimizing fingerprinting by passive attackers. The IETF is only one part of the solution. We can work on the standards that describe how systems interact. However, implementations, policy makers, operators and users will also play a role in securing the Internet against pervasive attacks.

Sam Hartman: Debian: Init Interfaces Our Users *and* Free Software

Recently, I ve been watching the Debian Schadenfreude related to init systems. For those who do not know, Debian is trying to decide whether Systemd or Upstart will be used to start software on Debian. There are two other options, although I think Systemd and Upstart are the main contenders. Currently the Technical Committee is considering the issue, although there s some very real chance that the entire project will get dragged through this swamp with the general resolution process. This is one of those discussions that proves that Debian truly is a community: if we didn t have something really strong holding us together we d never put up with something this painful. Between the accusations that our users now know the persecution European pagans faced at the hands of Christians as the Systemd priests drive all before them, a heated discussion of how the Debian voting system interacts with this issue, two failed calls for votes, a discussion of conflicts of interests, and a last-minute discussion of whether the matter had been appropriately brought before the Technical committee (and if so, under what authority), there s certainly schadenfreude to go around if you re into that sort of thing. However, through all this, the Technical Committee has been doing great work understanding the issues; it has been a pleasure to watch their hard work from the side lines. I d like to focus on one key question they ve found: how tightly can other software depend on the init system. Each init system offers some nice features. Upstart has an event triggering model. Systemd manages login sessions and at least in my opinion has a really nice service file format. Can I take advantage of these in my software. If I do, then users of my software might need to use a particular init system. Ian Jackson argues that we should give our users the choice of what init system to run. He reminds us that Debian is a community that supports diversity and diverse goals. We support multiple desktop systems, web browsers, etc. Wherever we can, we support people working on their goals and give developers the opportunity to make it work. When developers succeed, we give our users the opportunity to choose from among the options that have been developed. I think of Ian s argument as an appeal to part of our Social Contract. Clause 4 of the social contract begins:
Our priorities are our users and free software.
We will be guided by the needs of our users and the free software community. We will place their interests first in our priorities. We will support the needs of our users for operation in many different kinds of computing environments.
Ian is right that our users will be served by choice in init systems. However, this trade off needs to be balanced against the needs of the free software community. Diversity is an important goal but it should not come at the price of stagnation and innovation. If I want to avoid using init scripts because they don t provide restart on failure, because they are hard to write correctly, and because they don t provide access to modern security isolation techniques, I should be able to do that. If Systemd service files provide a superior solution, I should be able to work toward using them. If the desire for init system diversity shuts down my ability to find like-minded people who want to take advantage of improvements in init systems and work towards that goal, then we ve significantly damaged Debian as a forum for technical cooperation. Early in my work as a Debian Developer, Anthony Towns taught me an interesting principle. Those who do the work get significant influence over how and what gets done. Ian s right that we need to enable people to work towards choice in init systems. Those like-minded people need to be given a chance to find each other and pursue their goal. However the cost of success should be on the shoulders of those who value choice in init systems. It should not come at the cost of preventing people from depending on improvements in init systems. The best proposal I ve seen for balancing this is to enumerate stable interfaces that people want to use. Things like the logind interface or my favorite the service file interface. It needs to be possible to make another implementation of the interface. The interface needs to be stable enough that a dedicated team could have a chance of catching up with an implementation. At some point during the release cycle such interfaces would need to be frozen. However, I don t think it is reasonable to mandate that there are multiple implementations of the interface, only that there could be. The point is to give people a chance to work towards diversity in init systems, not to force it down peoples throats kicking and screaming until they leave the project or ignore our processes. Steve Langasek and Colin Watson seem to be working towards this. It s possible there s another approach besides interfaces. My main concern is the same as Ian s: maintain Debian as a forum for people to pursue their goals and work together. I suspect we see the conflict in these goals differently. I hope that we as a project can explore this conflict and figure out where we have common ground. I commit to exploring how we can work towards init choice in my frameworks; I ask those who prioritize init choice to commit to explore how we can take advantage of new features in their framework.

4 April 2013

Russ Allbery: Debian packaging of Git upstreams

Since there's a discussion of packaging software for Debian that uses Git upstream on Planet Debian right now, I wanted to weigh in and advocate for my current workflow for this situation, which I'm quite fond of. It's worth noting that I'm also upstream for quite a few of the packages I maintain, all in Git, and I use (almost) exactly the same structure for packaging my own software as for packaging anyone else's. So I have some experience with both sides of this. First off, I completely agree with Joey: if upstream is already using Git, there's no reason not to base the Debian packaging on the upstream repository, and many, many reasons to do so. One of the biggest advantages is that when repositories share a common basis and have been regularly merged, you can easily cherry-pick commits, which is wonderful for security releases and situations where you need a quick bug fix from an unreleased upstream branch. I make very heavy use of this when packaging OpenAFS. I do, however, like to base the Debian packaging on the released tarball, if for no other reason than that's the artifact that other people can more easily confirm. Yes, you can do the same thing with a Git tag, but the tarball is what upstream considers a release, so if one is available, I think it makes the most sense to base the packaging on it. I do this even for my own software. Thankfully, it's not that difficult to do both. Sam Hartman was the one who showed me this technique, and (after I used a manual script for some time for a couple of packages) Guido G nther incorporated the support into git-import-orig. The key idea is to still import the tarball into the upstream branch, but instead of making that import a simple commit, you make it a merge commit referencing the upstream release tag or commit from their Git repository. This means that you still get the exact contents of the release tarball on the upstream branch (and pristine-tar works as normal), but that branch is also based on the full upstream line of development. Therefore, so is your packaging branch (master or what have you) since you merge upstream into it. You can then charry-pick and take advantage of all of the normal Git features when following upstream development. This is dead simple to do with git-import-orig. Just add the upstream repository as a remote for your Git repository, make sure it's up to date with git fetch and you have the upstream tags, and then pass the flag --upstream-vcs-tag <tag> to git-import-orig whenever importing the upstream release tarball. git-import-orig will handle the construction of the merge commit for you and everything will just work, exactly like it normally does with git-buildpackage except with a more complete history. This support was added in git-buildpackage 0.6.0~git20120324, so it's available in unstable and testing. (I was going to update my notes on Debian packaging with Git to include this information before posting this, but I see that it will require some restructuring and quite a few changes to that document and I don't have time tonight. Hopefully soon.)

Next.