Search Results: "will"

15 August 2017

Dirk Eddelbuettel: #9: Compacting your Shared Libraries

Welcome to the nineth post in the recognisably rancid R randomness series, or R4 for short. Following on the heels of last week's post, we aim to look into the shared libraries created by R. We love the R build process. It is robust, cross-platform, reliable and rather predicatable. It. Just. Works. One minor issue, though, which has come up once or twice in the past is the (in)ability to fully control all compilation options. R will always recall CFLAGS, CXXFLAGS, ... etc as used when it was compiled. Which often entails the -g flag for debugging which can seriously inflate the size of the generated object code. And once stored in $ RHOME /etc/Makeconf we cannot on the fly override these values. But there is always a way. Sometimes even two. The first is local and can be used via the (personal) ~/.R/Makevars file (about which I will have to say more in another post). But something I have been using quite a bite lately uses the flags for the shared library linker. Given that we can have different code flavours and compilation choices---between C, Fortran and the different C++ standards---one can end up with a few lines. I currently use this which uses -Wl, to pass an the -S (or --strip-debug) option to the linker (and also reiterates the desire for a shared library, presumably superfluous):
SHLIB_CXXLDFLAGS = -Wl,-S -shared
SHLIB_CXX11LDFLAGS = -Wl,-S -shared
SHLIB_CXX14LDFLAGS = -Wl,-S -shared
SHLIB_FCLDFLAGS = -Wl,-S -shared
SHLIB_LDFLAGS = -Wl,-S -shared
Let's consider an example: my most recently uploaded package RProtoBuf. Built under a standard 64-bit Linux setup (Ubuntu 17.04, g++ 6.3) and not using the above, we end up with library containing 12 megabytes (!!) of object code:
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ ls -lh src/RProtoBuf.so
-rwxr-xr-x 1 edd edd 12M Aug 14 20:22 src/RProtoBuf.so
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ 
However, if we use the flags shown above in .R/Makevars, we end up with much less:
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ ls -lh src/RProtoBuf.so 
-rwxr-xr-x 1 edd edd 626K Aug 14 20:29 src/RProtoBuf.so
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ 
So we reduced the size from 12mb to 0.6mb, an 18-fold decrease. And the file tool still shows the file as 'not stripped' as it still contains the symbols. Only debugging information was removed. What reduction in size can one expect, generally speaking? I have seen substantial reductions for C++ code, particularly when using tenmplated code. More old-fashioned C code will be less affected. It seems a little difficult to tell---but this method is my new build default as I continually find rather substantial reductions in size (as I tend to work mostly with C++-based packages). The second option only occured to me this evening, and complements the first which is after all only applicable locally via the ~/.R/Makevars file. What if we wanted it affect each installation of a package? The following addition to its src/Makevars should do:
strippedLib: $(SHLIB)
        if test -e "/usr/bin/strip"; then /usr/bin/strip --strip-debug $(SHLIB); fi
.phony: strippedLib
We declare a new Makefile target strippedLib. But making it dependent on $(SHLIB), we ensure the standard target of this Makefile is built. And by making the target .phony we ensure it will always be executed. And it simply tests for the strip tool, and invokes it on the library after it has been built. Needless to say we get the same reduction is size. And this scheme may even pass muster with CRAN, but I have not yet tried. Lastly, and acknowledgement. Everything in this post has benefited from discussion with my former colleague Dan Dillon who went as far as setting up tooling in his r-stripper repository. What we have here may be simpler, but it would not have happened with what Dan had put together earlier.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

14 August 2017

Reproducible builds folks: Reproducible Builds: Weekly report #119

Here's what happened in the Reproducible Builds effort between Sunday July 30 and Saturday August 5 2017: Media coverage We were mentioned on Late Night Linux Episode 17, around 29:30. Packages reviewed and fixed, and bugs filed Upstream packages: Debian packages: Reviews of unreproducible packages 29 package reviews have been added, 72 have been updated and 151 have been removed in this week, adding to our knowledge about identified issues. 4 issue types have been updated: Weekly QA work During our reproducibility testing, FTBFS bugs have been detected and reported by: diffoscope development Version 85 was uploaded to unstable by Mattia Rizzolo. It included contributions from: as well as previous weeks' contributions, summarised in the changelog. There were also further commits in git, which will be released in a later version: Misc. This week's edition was written by Ximin Luo, Bernhard M. Wiedemann and Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Jamie McClelland: Diversity doesn't help the bottom line

A Google software engineer's sexist screed against diversity has been making the rounds lately. Most notable are the offensive and mis-guided statements about gender essentialism, which honestly make the thing hard to read at all. What seems lost in the hype, however, is that his primary point seems quite accurate. In short: If Google successfully diversified it's workforce, racial and gender tensions would increase not decrease, divisiveness would spread and, with all liklihood, Google could be damaged. Imagine what would happen if the thousands of existing, mostly male, white and Asian engineers, the majority of whom are convinced that they play no part in racism and sexism, were confronted with thousands of smart and ambitious women, African Americans and Latinos who were becoming their bosses, telling them to work in different ways, and taking "their" promotions. It would be a revolution! I'd love to see it. Google's bosses definitely do not. That's why none of the diversity programs at Google or any other major tech company are having any impact - because they are not designed to have an impact. They are designed to boost morale and make their existing engineers feel good about what they do. Google has one goal: to make money. And one strategy: to design software that that people want to use. One of their tactics that is highly effective is building tight knit groups of programmers who work well together. If the creation of hostile, racist and sexist environments is a by-product - well, it's not one that affects their bottom line. Would Google make better software with a more diverse group of engineers? Definitely! For one, if African American engineers were working on their facial recognition software, it's doubtful it would have mistaken people with black faces for gorillas. However, if the perceived improvement in software outweighed the risks of diversification, then Google would not waste any time on feel-good programs and trainings - they would simply build a jobs pipeline and change their job outreach programs to recruit substantially more female, African Americans and Latino candidates. In the end, this risk avoidance and failure to perceive the limitations of homogeneity is the achiles heel of corporate software design. Our challenge is to see what we can build outside the confines of corporate culture that prioritizes profits, production efficiency, and stability. What can we do with teams that are willing to embrace racial and gender tension, risk diviseveness and be willing to see benefits beyond releasing version 1.0?

Holger Levsen: 20170812-reproducible-policy

"packages should build reproducibly" - after 4 years this work of many is in debian-policy now This post was written roughly 44h ago and now that the fix for #844431 has been merged into the git master branch, I'm publishing it - hoping you'll enjoy this as much as I do! So today is the last (official) day of DebConf17 and it looks like #844431: "packages should build reproducibly" will be merged into debian-policy today! So I'm super excited, super happy, quite tired and a bit sad (DebConf is ending ) right now! :-) Four years ago Lunar held a BoF at DebConf13 which started the initiative in Debian. I only got involved in September 2014 with setting up continuous tests, rebuilding each package twice with some variations and then comparing the results using diffoscope, which back then was still called debbindiff and which we renamed as part of our efforts to make Reproducible Builds the norm in Free Software. Many people have worked on this, and I'm also very happy how visible this has been in our talk here yesterday. You people rock and I'm very thankful and proud to be part of this team. Thank you everyone! And please understand: we are not 94% done yet (which our reproducibility stats might have made you think), rather more like half done or so. We still need tools and processes to enable anyone to indepently verify that a given binary comes from the sources it is said to be coming, this will involve distributing .buildinfo files and providing user interfaces in APT and elsewhere. And probably also systematic rebuilds by us and other parties. And 6 or 7% of the archive are a lot of packages still, eg in Buster we currently still have 273 unreproducible key packages and for a large part we don't have patches yet. So there is still a lot of work ahead. This is what was added to debian-policy now:
Reproducibility
---------------
Packages should build reproducibly, which for the purposes of this
document [#]_ means that given
- a version of a source package unpacked at a given path;
- a set of versions of installed build dependencies;
- a set of environment variable values;
- a build architecture; and
- a host architecture,
repeatedly building the source package for the build architecture on
any machine of the host architecture with those versions of the build
dependencies installed and exactly those environment variable values
set will produce bit-for-bit identical binary packages.
It is recommended that packages produce bit-for-bit identical binaries
even if most environment variables and build paths are varied.  It is
intended for this stricter standard to replace the above when it is
easier for packages to meet it.
.. [#]
   This is Debian's precisification of the  reproducible-builds.org
   definition  _.
For now violating this part of policy may result in a severity: normal bug, though I think we should still only file them if we have patches, else it's probably better to just take a note in our notes.git, like we did before the policy change. Finally one last comment: we could do reproducible security updates for Stretch now too, for those 94% of the packages which are reproducible. It just needs to be done by someones and the first step would be publishing those .buildinfo files from those builds

Tim Retout: Jenkins milestone steps do not work yet

Public Service Announcement for anyone relying on Jenkins for continuous deployment - the milestone step plugin as of version 1.3.1 will not function correctly if you could have more than two builds running at once - older builds could get deployed after newer builds. See JENKINS-46097. A possible workaround is to add an initial milestone at the start of the pipeline, which will then allow builds to be killed early. (Builds are only killed early once they have passed their first milestone.) Going by the source history, I reckon this bug has been present since the milestone-step plugin was created.

13 August 2017

Lars Wirzenius: Retiring Obnam

This is a difficult announcement to write. The summary is if you use Obnam you should switch to another backup program in the coming months. The first commit to Obnam's current code base is this:
commit 7eaf5a44534ffa7f9c0b9a4e9ee98d312f2fcb14
Author: Lars Wirzenius <liw@iki.fi>
Date:   Wed Sep 6 18:35:52 2006 +0300
    Initial commit.
It's followed by over 5200 more commits until the latest one, which is from yesterday. The NEWS file contains 58 releases. There are 20761 lines of Python, 15384 words in the English language manual, with translations in German and French. The yarn test suite, which is a kind of a manual, is another 13382 words in English and pseudo-English. That's a fair bit of code and prose. Not all of it mine, I've had help from some wonderful people. But most of it mine. I wrote all of that because backups were fun. It was pleasing to use my own program to guarantee the safety of my own data. The technical challenges of implmenting the kind of backup program I wanted were interesting, and solving interesting problems is a big part of why I am a programmer. Obnam has a kind user base. It's not a large user base: the Debian "popularity contest" service estimates it at around 500. But it's a user base that is kind and has treated me well. I have tried to reciprocate. Unfortunately, I have not had fun while developing Obnam for some time now. This has changed. A few years ago, I lived in Manchester, UK, and commuted by train to work. It was a short train ride, about 15 minutes. At times I would sit on the floor with my laptop on my knees, writing code or the manual. Back then Obnam was a lot of fun. I was excited, and enthusiastic. In the past two years or so, I've not been able to feel that excitement again. My native language, Finnish, has an expression describing unpleasant tasks: something is as much fun as drinking tar. That describes Obnam in recent years for me. Obnam has not turned out well, from a maintainability point of view. It seems that every time I try to fix something, I break something else. Usuaully what breaks is speed or memory use: Obnam gets slower or starts using even more memory. For several years now I've been working on a new repository format for Obnam, code names GREEN ALBATROSS. It was meant to solve Obnam's problems as far as extensibility, performance, and resource use were concerned. It seems to have failed. I'm afraid I've had enough. I'm going to retire Obnam as a project and as a program, and move on to doing something else, so I can feel excitement and pleasure again. After some careful thought, I fear that the maintainability problems of Obnam can realistically only be solved by a complete rewrite from scratch, and I'm not up to doing that. If you use Obnam, you should migrate to some other backup solution. Don't worry, you have until the end of the year. I will be around and I intend to fix any serious bugs in Obnam; in particular, security flaws. But you should start looking for a replacement sooner rather than later. I will be asking Obnam to be removed from the Debian unstable and testing branches. The next Debian release (buster, Debian 10) won't include Obnam. The Obnam mailing lists are kindly hosted by Daniel Silverstone, and they will remain, but later this year I will change them to be moderated. The Obnam git repository will remain. The web site will remain, but I will add a note that Obnam is no longer maintained. Other Obnam online resources may disappear. If you would like to take over the Obnam project, and try to resolve the various issues, please contact me to discuss that. Thank you, and may you never need to restore.

Mike Gabriel: @DebConf17: Ad-hoc BoF: Debian for the Remote Desktop

On Thursday at DebConf17, all people interested in using this or that Remote Desktop solution on Debian (as a server, as a client, as both) came together for a BoF. Sharing about Usage Scenarios Quite some time we informally shared with one another what technologies and software we use for remote access to Debian machines and what the experiences are. The situation in Debian and on GNU/Linux in general is that many technical approaches exist, all of them have certain features and certain limitations. The composition of features and limitations finally lead the users to choosing one or another technology as his or her favourite solution. The Debian Remote Maintainers Team On the developers' side, Dominik George and I set up a packaging team for Remote Desktop related software in Debian. A packaging team that we invite everyone who is maintaining such software in the widest sense to join: https://qa.debian.org/developer.php?login=pkg-remote-team%40lists.alioth... 'DebianRemote' namespace on the Debian Wiki For users of Debian, the group agreed, we need an overview page (on wiki.debian.org) that provides an entry point for Debian on the Remote Desktop. An entry point that provides user information as well as developer information. A skeleton of this wiki page, I have just set up (thanks to Vagrant for taking some notes in Gobby during the BoF): https://wiki.debian.org/DebianRemote However, the page still contains loads of FIXMEs, so the actual work only now really starts. Fill the template with content (and also adapt the template, if needed). Everyone with experience and know-how about Remote Desktop on Debian systems is invited to share knowledge and improve this wiki namespace. (I will, at the earliest, start working on Arctica, X2Go and NX passages end of August, but I'll be also happy to find passages having been written down that I can review by then). Tracking Debian Remote Issues in Debian BTS At the BoF, also the following suggestions came up: The Remote Desktop experience on a GNU/Linux desktop or terminal server can be affected by all graphical applications available. Often it happens, that a change in this or that graphical application results in problems in remote sessions, but not so in local sessions. We agreed on filing and tagging such bugs accordingly. For new bugs, please file such bugs with the following BTS header at the top of your mail and always explain what remote desktop solution is being used when the bug appears:
Package: file
Version: 1:5.19-2
Severity: important
User: debian-edu@lists.debian.org
Usertags: debian-edu
Conclusion Overall, I was quite happy that the BoF has been attended by so many people and to see that there is quite "a lobby" in Debian. Let's dive into the work and make Debian 10 the first Debian, that mentions the Remote Desktop in its release notes. Let's, in fact, release Debian 10 as the first Debian with the official announcement as an operating system for the Remote Desktop (like the Fedora people did already for Fedora 20).

Enrico Zini: Consensually doing things together?

On 2017-08-06 I have a talk at DebConf17 in Montreal titled "Consensually doing things together?" (video). Here are the talk notes. Abstract At DebConf Heidelberg I talked about how Free Software has a lot to do about consensually doing things together. Is that always true, at least in Debian? I d like to explore what motivates one to start a project and what motivates one to keep maintaining it. What are the energy levels required to manage bits of Debian as the project keeps growing. How easy it is to say no. Whether we have roles in Debian that require irreplaceable heroes to keep them going. What could be done to make life easier for heroes, easy enough that mere mortals can help, or take their place. Unhappy is the community that needs heroes, and unhappy is the community that needs martyrs. I d like to try and make sure that now, or in the very near future, Debian is not such an unhappy community. Consensually doing things together I gave a talk in Heidelberg. Valhalla made stickers Debian France distributed many of them. There's one on my laptop. Which reminds me of what we ought to be doing. Of what we have a chance to do, if we play our cards right. I'm going to talk about relationships. Consensual relationships. Relationships in short. Nonconsensual relationships are usually called abuse. I like to see Debian as a relationship between multiple people. And I'd like it to be a consensual one. I'd like it not to be abuse. Consent From wikpedia:
In Canada "consent means the voluntary agreement of the complainant to engage in sexual activity" without abuse or exploitation of "trust, power or authority", coercion or threats.[7] Consent can also be revoked at any moment.[8] There are 3 pillars often included in the description of sexual consent, or "the way we let others know what we're up for, be it a good-night kiss or the moments leading up to sex." They are:
  • Knowing exactly what and how much I'm agreeing to
  • Expressing my intent to participate
  • Deciding freely and voluntarily to participate[20]
Saying "I've decided I won't do laundry anymore" when the other partner is tired, or busy doing things. Is different than saying "I've decided I won't do laundry anymore" when the other partner has a chance to say "why? tell me more" and take part in negotiation. Resources: Relationships Debian is the Universal Operating System. Debian is made and maintained by people. The long term health of debian is a consequence of the long term health of the relationship between Debian contributors. Debian doesn't need to be technically perfect, it needs to be socially healthy. Technical problems can be fixed by a healty community. graph showing relationship between avoidance, accomodation, compromise, competition, collaboration The Thomas-Kilmann Conflict Mode Instrument: source png. Motivations Quick poll: What are your motivations to be in a relationship? Which of those motivations are healthy/unhealthy? "Galadriel" (noun, by Francesca Ciceri): a task you have to do otherwise Sauron takes over Middle Earth See: http://blog.zouish.org/nonupdd/#/22/1 What motivates me to start a project or pick one up? What motivates me to keep maintaning a project? What motivates you? What's an example of a sustainable motivation? Is it really all consensual in Debian? Energy Energy that thing which is measured in spoons. The metaphore comes from people suffering with chronic health issues:
"Spoons" are a visual representation used as a unit of measure used to quantify how much energy a person has throughout a given day. Each activity requires a given number of spoons, which will only be replaced as the person "recharges" through rest. A person who runs out of spoons has no choice but to rest until their spoons are replenished.
For example, in Debian, I could spend: What is one person capable of doing? Have reasonable expectations, on others: Have reasonable expectations, on yourself: Debian is a shared responsibility When spoons are limited, what takes more energy tends not to get done As the project grows, project-wide tasks become harder Are they still humanly achievable? I don't want Debian to have positions that require hero-types to fill them Dictatorship of who has more spoons: Perfectionism You are in a relationship that is just perfect. All your friends look up to you. You give people relationship advice. You are safe in knowing that You Are Doing It Right. Then one day you have an argument in public. You don't just have to deal with the argument, but also with your reputation and self-perception shattering. One things I hate about Debian: consistent technical excellence. I don't want to be required to always be right. One of my favourite moments in the history of Debian is the openssl bug Debian doesn't need to be technically perfect, it needs to be socially healthy, technical problems can be fixed. I want to remove perfectionism from Debian: if we discover we've been wrong all the time in something important, it's not the end of Debian, it's the beginning of an improved Debian. Too good to be true There comes a point in most people's dating experience where one learns that when some things feel too good to be true, they might indeed be. There are people who cannot say no: There are people who cannot take a no: Note the diversity statement: it's not a problem to have one of those (and many other) tendencies, as long as one manages to keep interacting constructively with the rest of the community Also, it is important to be aware of these patterns, to be able to compensate for one's own tendencies. What happens when an avoidant person meets a narcissistic person, and they are both unaware of the risks? Resources: Note: there are problems with the way these resources are framed: Red flag / green flag http://pervocracy.blogspot.ca/2012/07/green-flags.html Ask for examples of red/green flags in Debian. Green flags: Red flags: Apologies / Dealing with issues I don't see the usefulness of apologies that are about accepting blame, or making a person stop complaining. I see apologies as opportunities to understand the problem I caused, help fix it, and possibly find ways of avoiding causing that problem again in the future. A Better Way to Say Sorry lists a 4 step process, which is basically what we do when in bug reports already: 1, Try to understand and reproduce the exact problem the person had. 2. Try to find the cause of the issue. 3. Try to find a solution for the issue. 4. Verify with the reporter that the solution does indeed fix the issue. This is just to say
My software ate
the files
that where in
your home directory and which
you were probably
needing
for work Forgive me
it was so quick to write
without tests
and it worked so well for me
(inspired by a 1934 poem by William Carlos Williams) Don't be afraid to fail Don't be afraid to fail or drop the ball. I think that anything that has a label attached of "if you don't do it, nobody will", shouldn't fall on anybody's shoulders and should be shared no matter what. Shared or dropped. Share the responsibility for a healthy relationship Don't expect that the more experienced mates will take care of everything. In a project with active people counted by the thousand, it's unlikely that harassment isn't happening. Is anyone writing anti-harassment? Do we have stats? Is having an email address and a CoC giving us a false sense of security?
When you get involved in a new community, such as Debian, find out early where, if that happens, you can find support, understanding, and help to make it stop. If you cannot find any, or if the only thing you can find is people who say "it never happens here", consider whether you really want to be in that community.
(from http://www.enricozini.org/blog/2016/debian/you-ll-thank-me-later/)
There are some nice people in the world. I mean nice people, the sort I couldn t describe myself as. People who are friends with everyone, who are somehow never involved in any argument, who seem content to spend their time drawing pictures of bumblebees on flowers that make everyone happy. Those people are great to have around. You want to hold onto them as much as you can. But people only have so much tolerance for jerkiness, and really nice people often have less tolerance than the rest of us. The trouble with not ejecting a jerk whether their shenanigans are deliberate or incidental is that you allow the average jerkiness of the community to rise slightly. The higher it goes, the more likely it is that those really nice people will come around less often, or stop coming around at all. That, in turn, makes the average jerkiness rise even more, which teaches the original jerk that their behavior is acceptable and makes your community more appealing to other jerks. Meanwhile, more people at the nice end of the scale are drifting away.
(from https://eev.ee/blog/2016/07/22/on-a-technicality/) Give people freedom If someone tries something in Debian, try to acknowledge and accept their work. You can give feedback on what they are doing, and try not to stand in their way, unless what they are doing is actually hurting you. In that case, try to collaborate, so that you all can get what you need. It's ok if you don't like everything that they are doing. I personally don't care if people tell me I'm good when I do something, I perceive it a bit like "good boy" or "good dog". I rather prefer if people show an interest, say "that looks useful" or "how does it work?" or "what do you need to deploy this?" Acknowledge that I've done something. I don't care if it's especially liked, give me the freedom to keep doing it. Don't give me rewards, give me space and dignity. Rather than feeding my ego, feed by freedom, and feed my possibility to create.

Mike Gabriel: @DebConf17: Work for Debian and FLOSS I got done during DebCamp and DebConf... and Beyond...

People I Met and will Remember Topics I have worked on Talks and BoFs Packages Uploaded to Debian unstable Packages Uploaded to Debian NEW I also looked into lightdm-webkit2-greeter, but upstream is in the middle of a transition from Gtk3 to Qt5, so this has been suspended for now. Packages Uploaded to oldstable-/stable-proposed-updates or -security Other Package related Stuff Thanks to Everyone Making This Event Possible A big thanks to everyone who made it possible for me to attend this event!!!

12 August 2017

Bits from Debian: DebConf17 closes in Montreal and DebConf18 dates announced

DebConf17 group photo - click to enlarge Today, Saturday 12 August 2017, the annual Debian Developers and Contributors Conference came to a close. With over 405 people attending from all over the world, and 169 events including 89 talks, 61 discussion sessions or BoFs, 6 workshops and 13 other activities, DebConf17 has been hailed as a success. Highlights included DebCamp with 117 participants, the Open Day,
where events of interest to a broader audience were offered, talks from invited speakers (Deb Nicholson, Matthew Garrett and Katheryn Sutter), the traditional Bits from the DPL, lightning talks and live demos and the announcement of next year's DebConf (DebConf18 in Hsinchu, Taiwan). The schedule has been updated every day, including 32 ad-hoc new activities, planned
by attendees during the whole conference. For those not able to attend, talks and sessions were recorded and live streamed, and videos are being made available at the Debian meetings archive website. Many sessions also facilitated remote participation via IRC or a collaborative pad. The DebConf17 website will remain active for archive purposes, and will continue to offer links to the presentations and videos of talks and events. Next year, DebConf18 will be held in Hsinchu, Taiwan, from 29 July 2018 until 5 August 2018. It will be the first DebConf held in Asia. For the days before DebConf the local organisers will again set up DebCamp (21 July - 27 July), a session for some intense work on improving the distribution, and organise the Open Day on 28 July 2018, aimed at the general public. DebConf is committed to a safe and welcome environment for all participants. See the DebConf Code of Conduct and the Debian Code of Conduct for more details on this. Debian thanks the commitment of numerous sponsors to support DebConf17, particularly our Platinum Sponsors Savoir-Faire Linux, Hewlett Packard Enterprise, and Google. About Savoir-faire Linux Savoir-faire Linux is a Montreal-based Free/Open-Source Software company with offices in Quebec City, Toronto, Paris and Lyon. It offers Linux and Free Software integration solutions in order to provide performance, flexibility and independence for its clients. The company actively contributes to many free software projects, and provides mirrors of Debian, Ubuntu, Linux and others. About Hewlett Packard Enterprise Hewlett Packard Enterprise (HPE) is one of the largest computer companies in the world, providing a wide range of products and services, such as servers, storage, networking, consulting and support, software, and financial services. HPE is also a development partner of Debian, and provides hardware for port development, Debian mirrors, and other Debian services. About Google Google is one of the largest technology companies in the world, providing a wide range of Internet-related services and products as online advertising technologies, search, cloud computing, software, and hardware. Google has been supporting Debian by sponsoring DebConf since more than ten years, at gold level since DebConf12, and at platinum level for this DebConf17.

Sam Hartman: Debian: a Commons of Innovation

I recently returned from Debconf. This year at Debconf, Matthew Garrett gave a talk about the next twenty years in free software. In his talk he raised concerns that Debian might not be relevant in that ecosystem and talked about some of the trends that contribute to his concerns.
I was talking to Marga after the talk and she said that Debian used to be a lot more innovative than it is today.
My initial reaction was doubt; what she said didn't feel right to me. At the time I didn't have a good answer. Since then I've been pondering the issue, and I think I have a partial answer to both Marga and Matthew and so I'll share it here.
In the beginning Debian focused on a lot of technical innovations related to bringing an operating system together. We didn't understand how to approach builds and build dependencies in a uniform manner. Producing packages in a clean environment was hard. We didn't understand what we wanted out of packages in terms of a uniform approach to configuration handling and upgrades. To a large extent we've solved those problems.
However, as the community has grown, our interests are more diverse. Our users and free software (and the operating system we build together) are what bring us together: we still have a central focus. However, no one technical project captures us all. There's still significant technical innovation in the Debian ecosystem. That innovation happens in Debian teams, companies and organizations that interact with the Debian community. We saw several talks about such innovation this year. I found the talk about ostree and flatpak interesting, especially because it focused on people in the broader Debian ecosystem valuing Debian along with some of the same technologies that Matthew is worried will undermine our relevance.
Matthew talked about how Debian ends up being a man-in-the-middle. We're between users and developers. we're between distributions and upstreams. Users are frustrated because we hold back the latest version of software they want from getting to them. Developers are frustrated because we present our users with old versions of their software configured not as they would like, combined with different dependencies than they expect.
All these weaknesses are real.
However, I think Debian-in-the-middle is our greatest strength both on the technical and social front.
I value Debian because I get a relatively uniform interface to the software I use. I can take one approach to reporting bugs whether they are upstream or Debian specific. I expect the software to behave in uniform ways with regard to things covered by policy. I know that I'm not going to have to configure multiple different versions of core dependencies; for the most part system services are shared. When Debian has value it's because our users want those things we provide. Debian has also reviewed every source file in the software we ship to understand the license and license compatibility. As a free software supporter and as someone who consumes software in commercial context, that value alone is enormous.
The world has evolved and we're facing technologies that provide different models. They've been coming for years: Python, Ruby, Java, Perl and others have been putting together their own commons of software. They have all been working to provide virtualization to isolate one program (and its dependencies) from another. Containerization takes that to the next level. Sometimes that's what our users want.
We haven't figured out what the balances are, how we fit into this new world. However, I disagree with the claim that we aren't even discussing the problem. I think we're trying a lot of things off in our own little technical groups. We're just getting to the level of critical mass of understanding where we can take advantage of Debian's modern form of innovation.
Because here's the thing. Debian's innovation now is social, not technical. Just as we're in the middle technically, we're in the middle socially. Upstreams, developers, users, derivatives, and all the other members of our community work together. we're a place where we can share technology, explore solutions, and pull apart common elements. This is the first Debconf where it felt like we'd explored some of these trends enough to start understanding how they might fit together in a whole. Seven years ago, it felt like we were busy being convinced the Java folks were wrong-headed. A couple of years later, it felt like we were starting to understand our users' desires that let to models different than packaging, but we didn't have any thoughts. At least in my part of the hallway it sounded like people were starting to think about how they might fit parts together and what the tradeoffs would be.
Yes, Matthew's talk doubtless sparked some of that. I think he gave us a well deserved and important wake-up call. However, I was excited by the discussion prior to Thursday.
What I'm taking a way is that Debian is valuable when there's a role in the middle. Both socially and technically we should capitalize on situations where something between makes things better and get out of the way where it does not.

11 August 2017

Thorsten Glaser: [PSA] Fixing CVE-2017-12836 (Debian #871810) in GNU cvs

Considering I ve become the de-facto upstream of cvs(GNU) even if not yet formally the de-iure upstream maintainer, fixing this bug obviously falls to me not quite the way I had planned passing this evening after coming home from work and a decent and, worse, very filling meal at the local Croatian restaurant. But, so s life. The problem here is basically that CVS invokes ssh(1) (well, rsh originally ) but doesn t add the argument separator -- before the (user-provided) hostname, which when starting with a hyphen-minus will be interpreted by ssh as an argument. (Apparently the other VCSes also had additional vulnerabilities such as not properly escaping semicoloi or pipes from the shell or unescaping percent-escaped fun characters, but that doesn t affect us.) The obvious fix and the one I implemented first is to simply add the dashes. This will also be backported to Debian , ,old old stable-security. Then I looked at other VCSes out of which only one did this, but they all added extra paranoia hostname checks (some of them passing invalid hostnames, such as those with underscores in them). OK, I thought, then also let s add extra checks to CVS repository reference handling. This will end up in Debian sid and MirBSD, pending passing the regression tests of course hah, while writing this article I had to fixup because a test failed. Anyway, it s not strictly necessary AFAICT to fix the issue. Update, about 2 hours past midnight (the testsuite runs for several hours): of course, the sanity testsuite (which itself is rather insane ) also needs adjustments, plus a bonus fix (for something that got broken when the recent allow-root-regex patch was merged and got fixed in the same go to night). tl;dr: a fix will end up in Debian *stable-security and can be taken out of my mail to the bugreport; another few changes for robustness are being tested and then added to both MirBSD and Debian sid. The impact is likely small, as it s hard to get a user (if you find one, in the first place) to use a crafted CVSROOT string, which is easy to spot as well. Update, Monday: apparently someone took care of the DSA and DLA yesterday after ACCEPTing the uploads thanks, I was outside during the day.

Mike Gabriel: @DebConf 2017: Ayatana Indicators

On last Tuesday, I gave a 20 min talk about Ayatana Indicators at DebConf 17 in Montreal. Ayatana Indicators Talk The talk had video coverage, so big thanks to the DebConf video team for making it possible to send the below video link around to people in the world: http://meetings-archive.debian.net/pub/debian-meetings/2017/debconf17/ay... The document of notes shown in the video is available on Debian's Infinote (Gobby) server:
$ sudo apt-get install gobby
$ sudo gobby infinote://gobby.debian.org/debconf17/talk/ayatana-indicators 
The major outcome of this talk was getting to know Dimitri John Ledkov from the Foundation Team at Canonical Ltd. We agreed on investigating the following actions, targetting the Ubuntu 18.04 LTS release and later on Debian 10 (aka buster): Upstream Todos Debian/Ubuntu Todos Please get in Touch... As this is going to be quite an effort, esp. if we want to get this done until 18.04 LTS, let me say, that this blog post is a call for help. If you are attached to Ubuntu and have used desktops with indicator support until now, please get in touch with the Ayatana Indicators team upstream as well as downstream (Debian/Ubuntu). Contact: Looking forward to meeting you online or on person and possibly working together with you on this transition project.

Mike Gabriel: @DebConf17: Story Telling about Debian Edu in Northern Germany

Last Monday, I gave a 20min talk about our little FLOSS school project "IT-Zukunft Schule" at the Debian Conference 17 in Montreal. The talk had video coverage, so may want to peek in, if you couldn't manage to watch the life stream: http://meetings-archive.debian.net/pub/debian-meetings/2017/debconf17/su... I'd like to share some major outcomes (so far) of this talk.
  1. I realized how attached I am to "IT-Zukunft Schule" and how much it means to me that our kids grow up in a world of freedom and choice. Also and esp. when it comes to choosing your daily communication tools and computer working environment
  2. I met Foteini Tsiami and Alkis Georgopoulos from Greece. They work on LTSP and have deployed 1000+ schools in Greece with LTSP + Debian GNU/Linux + MATE Desktop Environment
  3. I met Vagrant Cascadian who is the maintainer of LTSP in Debian and also a major LTSP upstream contributor
  4. I received a lot of fine feedback that was very encouraging to go on with our local work in Schleswig-Holstein
If you have some more time for watching DebConf talks on video, I dearly recommend the talk given by Alkis and Foteini on their Greek FLOSS success story. If you don't have that much time, please skip through the video until you are at 26:15 and enjoy the map that shows how much Debian + LTSP has spread over all of Greece. http://meetings-archive.debian.net/pub/debian-meetings/2017/debconf17/lt... Unfortunately, the schools in Greece are so much smaller than schools in Germany. Most schools there have between 50 and 300 students. So at the Greek schools, it is possible to have a teacher machine being the server for one computer lab. This teacher / server machine provides the infrastructure for a room full of LTSP fat clients (no hard drive inside) and that's it. For German schools, unfortunately, we need a larger scale setup. German schools often have 800+ students and network services need to be spread over more than one server machine. So, the current approach with one server running LDAP, Kerberos etc. is quite appropriate, but also extendible, possibly on municipality level or on county level. We (from IT-Zukunft Schule) are quite positive that there will be opportunities for introducing FLOSS approaches more on the county level in Schleswig-Holstein in the near future. So stay tuned...

10 August 2017

Kartik Mistry: DebConf17

So, I m here. It will take sometime to write about this, but this is to just reminder to myself that I exists on this blog too!

9 August 2017

Petter Reinholdtsen: Simpler recipe on how to make a simple $7 IMSI Catcher using Debian

On friday, I came across an interesting article in the Norwegian web based ICT news magazine digi.no on how to collect the IMSI numbers of nearby cell phones using the cheap DVB-T software defined radios. The article refered to instructions and a recipe by Keld Norman on Youtube on how to make a simple $7 IMSI Catcher, and I decided to test them out. The instructions said to use Ubuntu, install pip using apt (to bypass apt), use pip to install pybombs (to bypass both apt and pip), and the ask pybombs to fetch and build everything you need from scratch. I wanted to see if I could do the same on the most recent Debian packages, but this did not work because pybombs tried to build stuff that no longer build with the most recent openssl library or some other version skew problem. While trying to get this recipe working, I learned that the apt->pip->pybombs route was a long detour, and the only piece of software dependency missing in Debian was the gr-gsm package. I also found out that the lead upstream developer of gr-gsm (the name stand for GNU Radio GSM) project already had a set of Debian packages provided in an Ubuntu PPA repository. All I needed to do was to dget the Debian source package and built it. The IMSI collector is a python script listening for packages on the loopback network device and printing to the terminal some specific GSM packages with IMSI numbers in them. The code is fairly short and easy to understand. The reason this work is because gr-gsm include a tool to read GSM data from a software defined radio like a DVB-T USB stick and other software defined radios, decode them and inject them into a network device on your Linux machine (using the loopback device by default). This proved to work just fine, and I've been testing the collector for a few days now. The updated and simpler recipe is thus to
  1. start with a Debian machine running Stretch or newer,
  2. build and install the gr-gsm package available from http://ppa.launchpad.net/ptrkrysik/gr-gsm/ubuntu/pool/main/g/gr-gsm/,
  3. clone the git repostory from https://github.com/Oros42/IMSI-catcher,
  4. run grgsm_livemon and adjust the frequency until the terminal where it was started is filled with a stream of text (meaning you found a GSM station).
  5. go into the IMSI-catcher directory and run 'sudo python simple_IMSI-catcher.py' to extract the IMSI numbers.
To make it even easier in the future to get this sniffer up and running, I decided to package the gr-gsm project for Debian (WNPP #871055), and the package was uploaded into the NEW queue today. Luckily the gnuradio maintainer has promised to help me, as I do not know much about gnuradio stuff yet. I doubt this "IMSI cacher" is anywhere near as powerfull as commercial tools like The Spy Phone Portable IMSI / IMEI Catcher or the Harris Stingray, but I hope the existance of cheap alternatives can make more people realise how their whereabouts when carrying a cell phone is easily tracked. Seeing the data flow on the screen, realizing that I live close to a police station and knowing that the police is also wearing cell phones, I wonder how hard it would be for criminals to track the position of the police officers to discover when there are police near by, or for foreign military forces to track the location of the Norwegian military forces, or for anyone to track the location of government officials... It is worth noting that the data reported by the IMSI-catcher script mentioned above is only a fraction of the data broadcasted on the GSM network. It will only collect one frequency at the time, while a typical phone will be using several frequencies, and not all phones will be using the frequencies tracked by the grgsm_livemod program. Also, there is a lot of radio chatter being ignored by the simple_IMSI-catcher script, which would be collected by extending the parser code. I wonder if gr-gsm can be set up to listen to more than one frequency?

Joey Hess: unifying OS installation and configuration management

Three years ago, I realized that propellor (my configuration management system that is configured using haskell) could be used as an installer for Debian (or other versions of Linux). In propellor is d-i 2.0, I guessed it would take "a month and adding a few thousand lines of code". I've now taken that month, and written that code, and I presented the result at DebConf yesterday. I demoed propellor building a live Debian installation image, and then handed it off to a volenteer from the audience to play with its visual user interface and perform the installation. The whole demo took around 20 minutes, and ended with a standard Debian desktop installation. (Video) The core idea is to reuse the same configuration management system for several different purposes.
  1. Building a bootable disk image that can be used as both a live system and as an OS installer.
  2. Running on that live system, to install the target system. Which can just involve copying the live system to the target disk and then letting the configuration management system make the necessary changes to get from the live system configuration to the target system configuration.
  3. To support such things as headless arm boards, building customized images tuned for the target board and use case, that can then simply be copied to the board to install.
  4. Optionally, running on the installed system later, to futher customize it. Starting from the same configuration that produced the installed system in the first place.
There can be enourmous code reuse here, and improvements made for one of those will often benefit all the rest as well. Once everything is handled by configuration management, all user interface requirements become just a matter of editing the configuration. Including: That's the gist of it. Configuration management reused for installation and image building, and multiple editor interfaces to make it widely usable. I was glad, sitting in to a BoF session before my talk, that several people in Debian are already thinking along similar lines. And if Debian wanted to take this work and run with it, I'd be glad to assist as propellor's maintainer. But the idea is more important than the code and I hope my elaboration of it helps point a way if not the way. While what I've built installs Debian, little of it is Debian-specific. It would probably be easy to port it to Arch Linux, which propellor already supports. There are Linux-specific parts, so porting to FreeBSD would be harder, but propellor knows, at the type level which OSs properties support, which will ease porting. GuixSD and NixOS already use configuration management for installation, and were part of my inspiration. I've extended what they do in some ways (in other ways they remain far ahead).
The code is here. And here are some links to more details about what I built, and ideas encountered along the way:

8 August 2017

Wouter Verhelst: DebConf17 first videos published

Due to some technical issues, it took a slight bit longer than I'd originally expected; but the first four videos of the currently running DebConf 17 conference are available. Filenames are based on the talk title, so that should be reasonably easy to understand. I will probably add an RSS feed (like we've done for DebConf 16) to that place some time soon as well, but code for that still needs to be written. Meanwhile, we're a bit behind on the reviewing front, with (currently) 34 talks still needing review. If you're interested in helping out, please join the #debconf-video channel on OFTC and ask what you can do. This is something which you can do from home if you're interested, so don't be shy! We'd be happy for your help.

7 August 2017

Ben Hutchings: Debian LTS work, July 2017

I was assigned 15 hours of work by Freexian's Debian LTS initiative and worked 14 hours. I will carry over 1 hour to the next month. I prepared and released an update on the Linux 3.2 longterm stable branch (3.2.91), and started work on the next update. However, I didn't make any uploads to Debian this month.

Rapha&#235;l Hertzog: My Free Software Activities in July 2017

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donors (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I was allocated 12 hours but I only managed to work for 7 hours (due to vacation and unanticipated customer work). I gave back the remaining hours to the pool as I didn t want to carry them over for August which will be also short due to vacation (BTW I m not attending Debconf). I spent my 7 hours doing CVE triaging during the week where I was in charge of the LTS frontdesk (I committed 22 updates to the security tracker). I did publish DLA-1010-1 on vorbis-tools but the package update had been prepared by Petter Reinholdtsen. Misc Debian work zim. I published an updated package in experimental (0.67~rc2-2) with the upstream bug fixes on the current release candidate. The final version has been released during my vacation and I will soon upload it to unstable. Debian Handbook. I worked with Petter Reinholdtsen to finalize the paperback version of the Norwegian translation of the Debian Administrator s Handbook (still covering Debian 8 Jessie). It s now available. Bug reports. I filed a few bugs related to my Kali work. #868678: autopkgtest s setup-testbed script is not friendly to derivatives. #868749: aideinit fails with syntax errors when /etc/debian_version contains spaces. debian-installer. I submitted a few d-i patches that I prepared for a customer who had some specific needs (using the hd-media image to boot the installer from an ISO stored in an LVM logical volume). I made changes to debian-installer-utils (#868848), debian-installer (#868852), and iso-scan (#868859, #868900). Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Next.

Previous.