Search Results: "ana"

23 June 2016

Joey Hess: twenty years of free software -- part 4 ikiwiki-hosting

ikiwiki-hosting is a spin-off from ikiwiki. I wrote it to manage many ikiwiki instances for Branchable, and made it free software out of principle. While Branchable has not reached the point of providing much income, it's still running after 6 years. Ikiwiki-hosting makes it pretty easy to maintain it, and I host all of my websites there. A couple of other people have also found ikiwiki-hosting useful, which is not only nice, but led to some big improvements to it. Mostly though, releasing the software behind the business as free software caused us to avoid shortcuts and build things well. Next: twenty years of free software -- part 5 pristine-tar

22 June 2016

Scarlett Clark: KDE: Debian: *ubuntu snappy: Reproducible builds, Randa! and much more

#Randa2016 KDE Sprint

#Randa2016 KDE Sprint

Debian: I am very late on post due to travel, Flu, jetlag sorry! choqok:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=825322
For this I was able to come up with a patch for kconfig_compiler to encode generated files to utf-8.
Review request is here:
https://git.reviewboard.kde.org/r/128102/
This has been approved and I will be pushing it as soon as I patch the qt5 frameworks version.
Both kde4libs and kf5 kconfig has been pushed upstream kde. kxmlgui:
WIP this has been a steep learning curve, according to the notes it was an easy embedded kernel version, that was not the case! After grueling hours of
trying to sort out randomness in debug output I finally narrowed it down to cases where QStringLiteral was used and there were non letter characters eg. ( <") These were causing debug symbols to generate with ( lambda() ) which caused unreproducible symbol/debug files. It is now a case of fixing all of these in the code to use QString::fromUtf8 seems to fix this. I am working on a mega patch for upstream and it should be ready early in the week. This last week I spent a large portion making my through a mega patch for kxmlgui, when it was suggested to me to write a small qt app to test QStringLiteral isolated and sure enough two build were byte for byte identical. So this means that QStringLiteral may not be the issue at all. With some more assistance I am to expand my test app with several QStringLiterals of varying lengths, we have suspicion it is a padding issue, which complicates things.
I am still fighting with this one, will set aside to simmer for now, as I have no idea how to fix padding issues. extra-cmake-modules:
I am testing a patch to fix umask issues for anyone that uses the kapptemplate generation macro. Thank you Simon for pointing me to this.
known affected:
plasma-framework kdevplatform:
The kapptemplate generation users/groups and umask patch has been pushed upstream.
https://bugs.kde.org/show_bug.cgi?id=363615 KDE Randa!:
Despite managing to get a terrible Flu I accomplished more than I would have at home without awesome devs to help me out! CI TODO: Randa as usual was an amazing experience. Yes it is very hard work, but you have the beauty of the Swiss Alps at your fingertips! Not to mention all the
friendly faces and collaboration. A big thank you to all supporters and the Randa team! Please help make KDE better by supporting the very important Randa Sprint:
https://www.kde.org/fundraisers/randameetings2016/ Have a great day.

Andrew Cater: "But I'm a commercial developer / a government employee"

Following on:

Having seen some posts about this elsewhere on the 'Net:

  1. Your specific area of FLOSS expertise (or)
  2. Your time / hours in an area unrelated to your FLOSS expertise (or)
  3. A job that has no impact or bearing on your FLOSS expertise (or)
  4. Your time / hours only - and negotiate accordingly
Your employer may be willing to negotiate / grant you an opt-out clause to protect your FLOSS expertise / accept an additional non-exclusive licence to your FLOSS code / be prepared to sign an assignment e.g.

"You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright
interest in the program Gnomovision'
(which makes passes at compilers) written
by James Hacker.

signature of Ty Coon, 1 April 1989
Ty Coon, President of Vice"
 
[http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html

If none of the above is feasible: don't contribute anything that crosses the streams and mingles commercial and FLOSS expertise, however much you're offered to do so.

Patents / copyrights

"In the 1980s I had not yet realized how confusing it was to speak of the issue of intellectual property . That term is obviously biased; more subtle is the fact that it lumps together various disparate laws which raise very different issues. Nowadays I urge people to reject the term intellectual property entirely, lest it lead others to suppose that those laws form one coherent issue. The way to be clear is to discuss patents, copyrights, and trademarks separately. See further explanation of how this term spreads confusion and bias."
[http://www.gnu.org/gnu/manifesto.en.html - footnote 8.]

If you want to assert a patent - it's probably not FLOSS. Go away :)

If you want to assert a trademark of your own - it's probably not FLOSS. Go away :)
[Trademarks may ordinarily be outside the scope of normal FLOSS legal considerations - but should be acknowledged wherever they occur both as a matter of law and as a matter of courtesy]

Copyright gives legal standing (locus standi in the terminology of English common law) to sue for infringement - that's the basis of licence enforcement actions.

Employees of governments and those doing government work
This is complex: some FLOSS contributors see a significant amount of this as immaterial to them in the same way that some indigenous populations do not acknowledge imposed colonial legal structures as valid - but both value systems can co-exist




21 June 2016

Ian Wienand: Zuul and Ansible in OpenStack CI

In a prior post, I gave an overview of the OpenStack CI system and how jobs were started. In that I said
(It is a gross oversimplification, but for the purposes of OpenStack CI, Jenkins is pretty much used as a glorified ssh/scp wrapper. Zuul Version 3, under development, is working to remove the need for Jenkins to be involved at all).
Well some recent security issues with Jenkins and other changes has led to a roll-out of what is being called Zuul 2.5, which has indeed removed Jenkins and makes extensive use of Ansible as the basis for running CI tests in OpenStack. Since I already had the diagram, it seems worth updating it for the new reality.
OpenStack CI Overview While previous post was really focused on the image-building components of the OpenStack CI system, overview is the same but more focused on the launchers that run the tests. Overview of OpenStack CI with Zuul and Ansible
  1. The process starts when a developer uploads their code to gerrit via the git-review tool. There is no further action required on their behalf and the developer simply waits for results of their jobs.

  2. Gerrit provides a JSON-encoded "fire-hose" output of everything happening to it. New reviews, votes, updates and more all get sent out over this pipe. Zuul is the overall scheduler that subscribes itself to this information and is responsible for managing the CI jobs appropriate for each change.

  3. Zuul has a configuration that tells it what jobs to run for what projects. Zuul can do lots of interesting things, but for the purposes of this discussion we just consider that it puts the jobs it wants run into gearman for a launcher to consume. gearman is a job-server; as they explain it "[gearman] provides a generic application framework to farm out work to other machines or processes that are better suited to do the work". Zuul puts into gearman basically a tuple (job-name, node-type) for each job it wants run, specifying the unique job name to run and what type of node it should be run on.

  4. A group of Zuul launchers are subscribed to gearman as workers. It is these Zuul launchers that will consume the job requests from the queue and actually get the tests running. However, a launcher needs two things to be able to run a job a job definition (what to actually do) and a worker node (somewhere to do it). The first part what to do is provided by job-definitions stored in external YAML files. The Zuul launcher knows how to process these files (with some help from Jenkins Job Builder, which despite the name is not outputting XML files for Jenkins to consume, but is being used to help parse templates and macros within the generically defined job definitions). Each Zuul launcher gets these definitions pushed to it constantly by Puppet, thus each launcher knows about all the jobs it can run automatically. Of course Zuul also knows about these same job definitions; this is the job-name part of the tuple we said it put into gearman. The second part somewhere to run the test takes some more explaining. To the next point...

  5. Several cloud companies donate capacity in their clouds for OpenStack to run CI tests. Overall, this capacity is managed by a customized management tool called nodepool (you can see the details of this capacity at any given time by checking the nodepool configuration). Nodepool watches the gearman queue and sees what requests are coming out of Zuul. It looks at node-type of jobs in the queue (i.e. what platform the job has requested to run on) and decides what types of nodes need to start and which cloud providers have capacity to satisfy demand. Nodepool will start fresh virtual machines (from images built daily as described in the prior post), monitor their start-up and, when they're ready, put a new "assignment job" back into gearman with the details of the fresh node. One of the active Zuul launchers will pick up this assignment job and register the new node to itself.

  6. At this point, the Zuul launcher has what it needs to actually get jobs started. With an fresh node registered to it and waiting for something to do, the Zuul launcher can advertise its ability to consume one of the waiting jobs from the gearman queue. For example, if a ubuntu-trusty node is provided to the Zuul launcher, the launcher can now consume from gearman any job it knows about that is intended to run on an ubuntu-trusty node type. If you're looking at the launcher code this is driven by the NodeWorker class you can see this being created in response to an assignment via LaunchServer.assignNode. To actually run the job where the "job hits the metal" as it were the Zuul launcher will dynamically construct an Ansible playbook to run. This playbook is a concatenation of common setup and teardown operations along with the actual test scripts the jobs wants to run. Using Ansible to run the job means all the flexibility an orchestration tool provides is now available to the launcher. For example, there is a custom console streamer library that allows us to live-stream the console output for the job over a plain TCP connection, and there is the possibility to use projects like ARA for visualisation of CI runs. In the future, Ansible will allow for better coordination when running multiple-node testing jobs after all, this is what orchestration tools such as Ansible are made for! While the Ansible run can be fairly heavyweight (especially when you're talking about launching thousands of jobs an hour), the system scales horizontally with more launchers able to consume more work easily. When checking your job results on logs.openstack.org you will see a _zuul_ansible directory now which contains copies of the inventory, playbooks and other related files that the launcher used to do the test run.

  7. Eventually, the test will finish. The Zuul launcher will put the result back into gearman, which Zuul will consume (log copying is interesting but a topic for another day). The testing node will be released back to nodepool, which destroys it and starts all over again nodes are not reused and also have no sensitive details on them, as they are essentially publicly accessible. Zuul will wait for the results of all jobs for the change and post the result back to Gerrit; it either gives a positive vote or the dreaded negative vote if required jobs failed (it also handles merges to git, but that is also a topic for another day).

Work will continue within OpenStack Infrastructure to further enhance Zuul; including better support for multi-node jobs and "in-project" job definitions (similar to the https://travis-ci.org/ model); for full details see the spec.

Gunnar Wolf: Relax and breathe...

Time passes. I had left several (too many?) pending things to be done un the quiet weeks between the end of the lective semestre and the beginning of muy Summer trip to Winter. But Saturday gets closer every moment... And our long trip to the South begins. Among many other things, I wanted to avance with some Deb an stuff - both packaging and WRT keyring analysis. I want to contacto some people I left pending interactions with, but honestly, that will only come face to face un Capetown. As to "real life", I hace too many pending issues at work to even begin with; I hope to get some time at South frica todo do some decent UNAM sysadmining. Also, I want to play the idea of using Git for my students' workflow (handing in projects and assignments, at least)... This can be interesting to talk with the Deb an colleagues about, actually. As a Masters student, I'm making good advances, and will probably finish muy class work next semester, six months ahead of schedule, but muy thesis work so far has progressed way slower than what I'd like. I have at least a better defined topic and approach, so I'll start the writing phase soon. And the personal life? Family? I am more complete and happy than ever before. My life su completely different from two years ago. Yes, that was obvious. But it's also the only thing I can come up with. Having twin babies (when will they make the transition from "babies" to "kids"? No idea... We will find out as it comes) is more than beautiful, more than great. Our life has changed in every possible aspect. And yes, I admire my loved Regina for all of the energy and love she puts on the babies... Life is asymetric, I am out for most of the day... Mommy is always there. As I said, happier than ever before.

20 June 2016

Simon McVittie: GTK Hackfest 2016

I'm back from the GTK hackfest in Toronto, Canada and mostly recovered from jetlag, so it's time to write up my notes on what we discussed there. Despite the hackfest's title, I was mainly there to talk about non-GUI parts of the stack, and technologies that fit more closely in what could be seen as the freedesktop.org platform than they do in GNOME. In particular, I'm interested in Flatpak as a way to deploy self-contained "apps" in a freedesktop-based, sandboxed runtime environment layered over the Universal Operating System and its many derivatives, with both binary and source compatibility with other GNU/Linux distributions. I'm mainly only writing about discussions I was directly involved in: lots of what sounded like good discussion about the actual graphics toolkit went over my head completely :-) More notes, mostly from Matthias Clasen, are available on the GNOME wiki. In no particular order: Thinking with portals We spent some time discussing Flatpak's portals, mostly on Tuesday. These are the components that expose a subset of desktop functionality as D-Bus services that can be used by contained applications: they are part of the security boundary between a contained app and the rest of the desktop session. Android's intents are a similar concept seen elsewhere. While the portals are primarily designed for Flatpak, there's no real reason why they couldn't be used by other app-containment solutions such as Canonical's Snap. One major topic of discussion was their overall design and layout. Most portals will consist of a UX-independent part in Flatpak itself, together with a UX-specific implementation of any user interaction the portal needs. For example, the portal for file selection has a D-Bus service in Flatpak, which interacts with some UX-specific service that will pop up a standard UX-specific "Open" dialog for GNOME and probably other GTK environments, that dialog is in (a branch of) GTK. A design principle that was reiterated in this discussion is that the UX-independent part should do as much as possible, with the UX-specific part only carrying out the user interactions that need to comply with a particular UX design (in the GTK case, GNOME's design). This minimizes the amount of work that needs to be redone for other desktop or embedded environments, while still ensuring that the other environments can have their chosen UX design. In particular, it's important that, as much as possible, the security- and performance-sensitive work (such as data transport and authentication) is shared between all environments. The aim is for portals to get the user's permission to carry out actions, while keeping it as implicit as possible, avoiding an "are you sure?" step where feasible. For example, if an application asks to open a file, the user's permission is implicitly given by them selecting the file in the file-chooser dialog and pressing OK: if they do not want this application to open a file at all, they can deny permission by cancelling. Similarly, if an application asks to stream webcam data, the UX we expect is for GNOME's Cheese app (or a similar non-GNOME app) to appear, open the webcam to provide a preview window so they can see what they are about to send, but not actually start sending the stream to the requesting app until the user has pressed a "Start" button. When defining the API "contracts" to be provided by applications in that situation, we will need to be clear about whether the provider is expected to obtain confirmation like this: in most cases I would anticipate that it is. One security trade-off here is that we have to have a small amount of trust in the providing app. For example, continuing the example of Cheese as a webcam provider, Cheese could (and perhaps should) be a contained app itself, whether via something like Flatpak, an LSM like AppArmor or both. If Cheese is compromised somehow, then whenever it is running, it would be technically possible for it to open the webcam, stream video and send it to a hostile third-party application. We concluded that this is an acceptable trade-off: each application needs to be trusted with the privileges that it needs to do its job, and we should not put up barriers that are easy to circumvent or otherwise serve no purpose. The main (only?) portal so far is the file chooser, in which the contained application asks the wider system to show an "Open..." dialog, and if the user selects a file, it is returned to the contained application through a FUSE filesystem, the document portal. The reference implementation of the UX for this is in GTK, and is basically a GtkFileChooserDialog. The intention is that other environments such as KDE will substitute their own equivalent. Other planned portals include: Environment variables GNOME on Wayland currently has a problem with environment variables: there are some traditional ways to set environment variables for X11 sessions or login shells using shell script fragments (/etc/X11/Xsession.d, /etc/X11/xinit/xinitrc.d, /etc/profile.d), but these do not apply to Wayland, or to noninteractive login environments like cron and systemd --user. We are also keen to avoid requiring a Turing-complete shell language during session startup, because it's difficult to reason about and potentially rather inefficient. Some uses of environment variables can be dismissed as unnecessary or even unwanted, similar to the statement in Debian Policy 9.9: "A program must not depend on environment variables to get reasonable defaults." However, there are two common situations where environment variables can be necessary for proper OS integration: search-paths like $PATH, $XDG_DATA_DIRS and $PYTHONPATH (particularly necessary for things like Flatpak), and optionally-loaded modules like $GTK_MODULES and $QT_ACCESSIBILITY where a package influences the configuration of another package. There is a stopgap solution in GNOME's gdm display manager, /usr/share/gdm/env.d, but this is gdm-specific and insufficiently expressive to provide the functionality needed by Flatpak: "set XDG_DATA_DIRS to its specified default value if unset, then add a couple of extra paths". pam_env comes closer PAM is run at every transition from "no user logged in" to "user can execute arbitrary code as themselves" but it doesn't support .d fragments, which are required if we want distribution packages to be able to extend search paths. pam_env also turns off per-user configuration by default, citing security concerns. I'll write more about this when I have a concrete proposal for how to solve it. I think the best solution is probably a PAM module similar to pam_env but supporting .d directories, either by modifying pam_env directly or out-of-tree, combined with clarifying what the security concerns for per-user configuration are and how they can be avoided. Relocatable binary packages On Windows and OS X, various GLib APIs automatically discover where the application binary is located and use search paths relative to that; for example, if C:\myprefix\bin\app.exe is running, GLib might put C:\myprefix\share into the result of g_get_system_data_dirs(), so that the application can ask to load app/data.xml from the data directories and get C:\myprefix\share\app\data.xml. We would like to be able to do the same on Linux, for example so that the apps in a Flatpak or Snap package can be constructed from RPM or dpkg packages without needing to be recompiled for a different --prefix, and so that other third-party software packages like the games on Steam and gog.com can easily locate their own resources. Relatedly, there are currently no well-defined semantics for what happens when a .desktop file or a D-Bus .service file has Exec=./bin/foo. The meaning of Exec=foo is well-defined (it searches $PATH) and the meaning of Exec=/opt/whatever/bin/foo is obvious. When this came up in D-Bus previously, my assertion was that the meaning should be the same as in .desktop files, whatever that is. We agreed to propose that the meaning of a non-absolute path in a .desktop or .service file should be interpreted relative to the directory where the .desktop or .service file was found: for example, if /opt/whatever/share/applications/foo.desktop says Exec=../../bin/foo, then /opt/whatever/bin/foo would be the right thing to execute. While preparing a mail to the freedesktop and D-Bus mailing lists proposing this, I found that I had proposed the same thing almost 2 years ago... this time I hope I can actually make it happen! Flatpak and OSTree bug fixing On the way to the hackfest, and while the discussion moved to topics that I didn't have useful input on, I spent some time fixing up the Debian packaging for Flatpak and its dependencies. In particular, I did my first upload as a co-maintainer of bubblewrap, uploaded ostree to unstable (with the known limitation that the grub, dracut and systemd integration is missing for now since I haven't been able to test it yet), got most of the way through packaging Flatpak 0.6.5 (which I'll upload soon), cherry-picked the right patches to make ostree compile on Debian 8 in an effort to make backports trivial, and spent some time disentangling a flatpak test failure which was breaking the Debian package's installed-tests. I'm still looking into ostree test failures on little-endian MIPS, which I was able to reproduce on a Debian porterbox just before the end of the hackfest. OSTree + Debian I also had some useful conversations with developers from Endless, who recently opened up a version of their OSTree build scripts for public access. Hopefully that information brings me a bit closer to being able to publish a walkthrough for how to deploy a simple Debian derivative using OSTree (help with that is very welcome of course!). GTK life-cycle and versioning The life-cycle of GTK releases has already been mentioned here and elsewhere, and there are some interesting responses in the comments on my earlier blog post. It's important to note that what we discussed at the hackfest is only a proposal: a hackfest discussion between a subset of the GTK maintainers and a small number of other GTK users (I am in the latter category) doesn't, and shouldn't, set policy for all of GTK or for all of GNOME. I believe the intention is that the GTK maintainers will discuss the proposals further at GUADEC, and make a decision after that. As I said before, I hope that being more realistic about API and ABI guarantees can avoid GTK going too far towards either of the possible extremes: either becoming unable to advance because it's too constrained by compatibility, or breaking applications because it isn't constrained enough. The current situation, where it is meant to be compatible within the GTK 3 branch but in practice applications still sometimes break, doesn't seem ideal for anyone, and I hope we can do better in future. Acknowledgements Thanks to everyone involved, particularly:

Reproducible builds folks: Reorganizing the autopkgtest code

Author: ceridwen The very-first alpha-quality release of reprotest is now in Debian's NEW package queue, so it should soon be available for install through apt. I've spent most of the past week reverse-engineering autopkgtest, with some help from the maintainer, Martin Pitt. As its name suggests, autopkgtest is a tool for running Debian/Ubuntu package tests. The key overlap with reprotest is that it can create containers (what the autopkgtest documentation calls "virtualization servers" or "virt-servers") and then run tests in them. While reprotest builds packages and diffs binaries instead, the process of copying source trees and executing commands is much the same, so I can reuse a lot of the code. As I outlined last week, autopkgtest is composed of a main command-line program and other command-line programs chroot, lxd, schroot, null, schroot, and qemu, which live in a directory virt/. (I've followed autopkgtest 4.0 in dropping the adt-virt- prefix and other name conventions.) Autopkgtest's directory structure looks like:
autopkgtest/
    lib/
        adt_testbed.py
        adtlog.py
        VirtSubproc.py
    runner/
    setup-commands/
    ssh-setup/
    tests/
    tools/
    virt/
        chroot
        lxc
        lxd
        null
        qemu
        schroot
        ssh
Reprotest's directory structure looks like:
reprotest/
    reprotest/
    __init__.py
        lib/
            adt_testbed.py
            adtlog.py
            VirtSubproc.py
        runner/
        tools/
        virt/
            chroot
            lxc
            lxd
            null
            qemu
            schroot
            ssh
    tests/
Autopkgtest doesn't contain any __init__.py files, which means that from the perspective of Python none of its components are Python packages. To enable the various components of autopkgtest test to import each other as implicit namespace packages, there's a lot of explicit sys.path mutation code in the various files like sys.path.insert(0, '/usr/share/autopkgtest/python') and sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'lib')). For my design goals, there are two problems with this. First, setuptools makes it very, very difficult to distribute code that isn't in Python packages, so this approach more or less doesn't work with PyPi at all. Second, different OSes have different directory layouts for Python packages, so an absolute path like /usr/share/autopkgtest/python isn't portable. The main code of reprotest at the moment lives in its __init__.py. The autopkgtest CLI of the main runner is specific to running tests, so my entry point is going to be adt_testbed. My first attempt to refactor this involved putting an empty __init__.py in lib/ to turn into a package so that I could import the modules in the files in lib/ using code like from reprotest.lib import adtlog. This works fine for the files in lib/ and makes adt_testbed importable in the main __init__.py. However, the executables in virt/, which are invoked by subprocess calls in adt_testbed.py, also need to import modules from lib/ for the various adtlog functions and the timeout functions in VirtSubproc. After some trial-and-error, by replacing the sys.path mutations in virt/ with sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))), I managed to get the executables to run, though I still need to verify that this works for both PyPi and apt. I'm not sure if this is the best solution, so I may end up doing some kind of larger refactoring, either changing all the subprocess calls in adt_testbed so they work with reprotest.virt as a subpackage or reorganizing the rest of the code somehow. It's still not clear to me why autopkgtest uses its own special logging module rather than the standard library's logging module. I thought it might have something to do with combining the logging from a process and its subprocesses, but because the Python subprocesses are importing their own instances of the adtlog module, this isn't the case. Another problem related to logging and debugging is that the programs in virt/ interact through stdin and stdout, which means that I haven't been able to use simple prints to help work out their state during execution. Now that I have adt_testbed working, my next step is to change the code I've already written so that instead of executing commands directly, it executes them through the autopkgtest interface.

Daniel Pocock: WebRTC and communications projects in GSoC 2016

This year a significant number of students are working on RTC-related projects as part of Google Summer of Code, under the umbrella of the Debian Project. You may have already encountered some of them blogging on Planet or participating in mailing lists and IRC. WebRTC plugins for popular CMS and web frameworks There are already a range of pseudo-WebRTC plugins available for CMS and blogging platforms like WordPress, unfortunately, many of them are either not releasing all their source code, locking users into their own servers or requiring the users to download potentially untrustworthy browser plugins (also without any source code) to use them. Mesut is making plugins for genuinely free WebRTC with open standards like SIP. He has recently created the WPCall plugin for WordPress, based on the highly successful DruCall plugin for WebRTC in Drupal. Keerthana has started creating a similar plugin for MediaWiki. What is great about these plugins is that they don't require any browser plugins and they work with any server-side SIP infrastructure that you choose. Whether you are routing calls into a call center or simply using them on a personal blog, they are quick and convenient to install. Hopefully they will be made available as packages, like the DruCall packages for Debian and Ubuntu, enabling even faster installation with all dependencies. Would you like to try running these plugins yourself and provide feedback to the students? Would you like to help deploy them for online communities using Drupal, WordPress or MediaWiki to power their web sites? Please come and discuss them with us in the Free-RTC mailing list. You can read more about how to run your own SIP proxy for WebRTC in the RTC Quick Start Guide. Finding all the phone numbers and ham radio callsigns in old emails Do you have phone numbers and other contact details such as ham radio callsigns in old emails? Would you like a quick way to data-mine your inbox to find them and help migrate them to your address book? Jaminy is working on Python scripts to do just that. Her project takes some inspiration from the Telify plugin for Firefox, which detects phone numbers in web pages and converts them to hyperlinks for click-to-dial. The popular libphonenumber from Google, used to format numbers on Android phones, is being used to help normalize any numbers found. If you would like to test the code against your own mailbox and address book, please make contact in the #debian-data channel on IRC. A truly peer-to-peer alternative to SIP, XMPP and WebRTC The team at Savoir Faire Linux has been busy building the Ring softphone, a truly peer-to-peer solution based on the OpenDHT distribution hash table technology. Several students (Simon, Olivier, Nicolas and Alok) are actively collaborating on this project, some of them have been fortunate enough to participate at SFL's offices in Montreal, Canada. These GSoC projects have also provided a great opportunity to raise Debian's profile in Montreal ahead of DebConf17 next year. Linux Desktop Telepathy framework and reSIProcate Another group of students, Mateus, Udit and Balram have been busy working on C++ projects involving the Telepathy framework and the reSIProcate SIP stack. Telepathy is the framework behind popular softphones such as GNOME Empathy that are installed by default on the GNU/Linux desktop. I previously wrote about starting a new SIP-based connection manager for Telepathy based on reSIProcate. Using reSIProcate means more comprehensive support for all the features of SIP, better NAT traversal, IPv6 support, NAPTR support and TLS support. The combined impact of all these features is much greater connectivity and much greater convenience. The students are extending that work, completing the buddy list functionality, improving error handling and looking at interaction with XMPP. Streamlining provisioning of SIP accounts Currently there is some manual effort for each user to take the SIP account settings from their Internet Telephony Service Provider (ITSP) and transpose these into the account settings required by their softphone. Pranav has been working to close that gap, creating a JAR that can be embedded in Java softphones such as Jitsi, Lumicall and CSipSimple to automate as much of the provisioning process as possible. ITSPs are encouraged to test this client against their services and will be able to add details specific to their service through Github pull requests. The project also hopes to provide streamlined provisioning mechanisms for privately operated SIP PBXes, such as the Asterisk and FreeSWITCH servers used in small businesses. Improving SIP support in Apache Camel and the Jitsi softphone Apache Camel's SIP component and the widely known Jitsi softphone both use the JAIN SIP library for Java. Nik has been looking at issues faced by SIP users in both projects, adding support for the MESSAGE method in camel-sip and looking at why users sometimes see multiple password prompts for SIP accounts in Jitsi. If you are trying either of these projects, you are very welcome to come and discuss them on the mailing lists, Camel users and Jitsi users. GSoC students at DebConf16 and DebConf17 and other events Many of us have been lucky to meet GSoC students attending DebConf, FOSDEM and other events in the past. From this year, Google now expects the students to complete GSoC before they become eligible for any travel assistance. Some of the students will still be at DebConf16 next month, assisted by the regular travel budget and the diversity funding initiative. Nik and Mesut were already able to travel to Vienna for the recent MiniDebConf / LinuxWochen.at As mentioned earlier, several of the students and the mentors at Savoir Faire Linux are based in Montreal, Canada, the destination for DebConf17 next year and it is great to see the momentum already building for an event that promises to be very big. Explore the world of Free Real-Time Communications (RTC) If you are interesting in knowing more about the Free RTC topic, you may find the following resources helpful: RTC mentoring team 2016 We have been very fortunate to build a large team of mentors around the RTC-themed projects for 2016. Many of them are first time GSoC mentors and/or new to the Debian community. Some have successfully completed GSoC as students in the past. Each of them brings unique experience and leadership in their domain. Helping GSoC projects in 2016 and beyond Not everybody wants to commit to being a dedicated mentor for a GSoC student. In fact, there are many ways to help without being a mentor and many benefits of doing so. Simply looking out for potential applicants for future rounds of GSoC and referring them to the debian-outreach mailing list or an existing mentor helps ensure we can identify talented students early and design projects around their capabilities and interests. Testing the projects on an ad-hoc basis, greeting the students at DebConf and reading over the student wikis to find out where they are and introduce them to other developers in their area are all possible ways to help the projects succeed and foster long term engagement. Google gives Debian a USD $500 grant for each student who completes a project successfully this year. If all 2016 students pass, that is over $10,000 to support Debian's mission.

Valerie Young: Week 2 on Reproducible Builds

Here is a small update for last week s work: 1. Templates tests.reproducible-builds.org now has a templating system, instead of writing HTML in python files  We are using mustache with pystache. So far, only the html for the package pages have been converted. 2. Navigation improvements on package pages While converting the package pages to using mustache I also rearranged the navigation bar, adding sections and hover text. Navigation improvements inspired by Gerts Wollney s email requests. Please check out the changes and provide feedback: tests.reproducible-builds.org/<your-favorite-package> 3. Began converting script from bash to python Before converting the rest of the site to mustache, I spent a day digging into the last unconverted bash script (all other html producing scripts are python). I ve never written or read that much bash, yet another learning opportunity I have to admit it s a bit less intuitive than python  Up next: Finish converting package set script, create more mustache templates, more site improvements! Then fly to South Africa.

Keerthana Krishnan: 10 Git commands every developer should know

Git is a version control system which is very common among developers. I m not going to explain how git works, because there are plenty of great introductory tutorials on the internet for that. However, I would like to documment the commands I frequently use 1. First, we initialize a git repo. Take care to ensure that you initialise the repo in the folder where all the files you want to commit exists. git init 2. To see all the branches in the current repo along with the current branch : git branch To create a new branch : git branch <branch_name> 3. To change branches : git checkout <branch_name> If the branch doesn t exist, it can be created by using : git checkout -b <branch_name> 4. To see the status of the files, ie. if there are uncommitted changes, files to save etc.. use : git status 5. To stage the changes in the directory for the next commit, use : git add <filename> To stage all files in a directory git add . 6. To commit the staged changes in the directory, use : git commit -m "<commit_message>" 7. To create a new connection to a remote repo, especially to push existing code to a Github repo : git remote add <connection_name> <connection_url> 8. To push the code into the remote repo git push -u <connection_name> <branch_name> 9. To get the contents of a repo git clone <path> 10. To merge another branch to the current (Example: a side branch to the master) git merge <branch_name> For a more detailed list of git commands for easy access check out this nifty cheat sheet. Happy coding!

19 June 2016

Iustin Pop: Short trip to Seattle area

After last week's bike ride, I had to pack my bags, and get on Monday morning on a plane to Seattle. The time was so short that I even left the bike mounted on the car. So 8:15, plane to Frankfurt, and then plane to Seattle. To my big surprise, the Lufthansa "extended leg room" seats were overly generous; I could actually extend my foot completely and put it on the back of the seat in front of me. Very good value when travelling in economy The only downside was that these were "standard" not "premium" economy, so the seat had leg room but was very narrow. And with normal sized adults on either side of me, it was somewhat difficult. The leg space allowed me to work on my laptop without fearing the person in front of me will recline their seat and break my screen (almost happened once). The funniest thing when travelling is that food is always tricky: even familiar food can be not what you expect. Case in point, me at the salad buffet, seeing slices of green vegetables, and asking myself: Are those jalapeno slices, or bell pepper slices? Hmm, I'm sure they're bell pepper , which resulted in my first vegetable salad with jalapeno. Would definitely recommend if you like spicy things! Otherwise the trip was as usual, but shorter and more densely packed with meetings with a one day exception: had the opportunity to experience for the first time Whirlyball, which was more fun and more difficult than it first looked. Also spent an afternoon on Whidbey Island, which fortunately was also the nicest day, weather-wise, of the week (all phone pictures, not colour corrected, straight-out-of-phone): from the ferry at the end of Hobbit Trail before dinner A few more pictures here. I keep being amazed by the nature in this area, definitely my preferred place in US from the relatively few I visited. Had a nice dinner as well at Cafe Langley, which surprisingly had reasonably-authentic Mediterranean food; the Baba Ganoush was excellent. Other than that outing, nothing worth describing, except that I really missed my Zwift or outdoor rides. The experience of using the stationary bikes in the hotel does not compare, so I resorted more to running on the treadmill (hmm, Zwift for running, hmm ); if the foot pod calibration is to be trusted, I continued to slightly improve my 1K, 1mi, and 5K times. Not bad, I might want to join some running races as well this summer, but I need to take it easy though and make sure to not get injured again. And finally, week over, flew back home, slept a bit mid-day (which will ruin my jet lag recovery program), unloaded my bike from the car (and checked it still works), and and and got on the trainer and did a Zwift ride. Jet lagged, but managed to beat my Watopia sprint record by a tiny bit, and complete a new workout ( The Gorby ), which was interesting. I can stop any time I want, definitely (I just need to take a trip away from home ).

Keerthana Krishnan: What are Debian Backports?

I came across Debian Backports recently after a kind soul suggested it in my Jessie Installation article. Backports provide the bridge between two stable releases of Debian. Say you are running Jessie stable in your system now. While it runs great, Jessie stable will be a bit outdated compared to the testing release. This is solved by Backports which recompiles and adjusts these testing packages for the stable environment. Backports can be included into the source file next to the stable sources unlike the testing sources which can misbehave and have a significant risk, Backports are much more compatible and lower risk. However, Backports cannot be tested as extensively as Debian stable, and backports are provided on an as-is basis, with risk of incompatibilities with other components in Debian stable. Use with care! It is therefore recommended to only select single backported packages that fit your needs, and not use all available backports. To add Backports to your system, Open the sources.list file :
sudo nano /etc/apt/sources.list For jessie add the line :
deb http://ftp.debian.org/debian jessie-backports main For wheezy add the line :
deb http://ftp.debian.org/debian wheezy-backports main After any change be sure to run
sudo apt-get update To know more about Backports and to contribute consult the main page here and the wiki here. Happy hacking!

18 June 2016

Manuel A. Fernandez Montecelo: More work on aptitude

The last few months have been a bit of a crazy period of ups and downs, with a tempest of events beneath the apparent and deceivingly calm surface waters of being unemployed (still at it). The daily grind Chief activities are, of course, those related to the daily grind of job-hunting, sending applications, and preparing and attending interviews. It is demoralising when one searches for many days or weeks without seeing anything suitable for one's skills or interests, or other more general life expectations. And it takes a lot of time and effort to put one's best in the applications for positions that one is really, really, interested in. And even for the ones which are meh, for a variety of reasons (e.g. one is not very suitable for what the offer demands). After that, not being invited to interviews (or doing very badly at them) is bad, of course, but quick and not very painful. A swift, merciful end to the process. But it's all the more draining when waiting for many weeks when not a few months with the uncertainty of not knowing if one is going to be lucky enough to be summoned for an interview; harbouring some hope one has to appear enthusiastic in the interviews, after all , while trying to keep it contained lest it grows too much ; then in the interview hearing good words and some praises, and feeling the impression that one will fit in, that one did nicely and that chances are good letting the hope grow again ; start to think about life changes that the job will require to make a quick decision should the offer finally arrives ; perhaps make some choices and compromises based on the uncertain result; then wait for a week or two after the interview to know the result... ... only to end up being unsuccessful. All the effort and hopes finally get squashed with a cold, short email or automatic response, or more often than not, complete radio silence from prospective employers, as an end to a multi-month-long process. An emotional roller coaster [1], which happened to me several times in the last few months. All in a day's work The months of preparing and waiting for a new job often imply an impasse that puts many other things that one cares about on hold, and one makes plans that will never come to pass. All in a day's (half-year's?) work of an unemployed poor soul. But not all is bad. This period was also a busy time doing some plans about life, mid- and long-term; the usual and some really unusual! family events; visits to and from friends, old and new; attending nice little local Debian gatherings or the bigger gathering of Debian SunCamp2016, and other work for side projects or for other events that will happen soon... And amidst all that, I managed to get some work done on aptitude. Two pictures worth (less than) a thousand bugs To be precise, worth 709 bugs 488 bugs in the first graph, plus 221 in the second. In 2015-11-15 (link to the post Work on aptitude): aptitude BTS Graph, 2015-11-15 In 2016-06-18: aptitude BTS Graph, 2016-06-18 Numbers The BTS numbers for aptitude right now are: Highlights Beyond graphs and stats, I am specially happy about two achievements in the last year:
  1. To have aptitude working today, first and foremost Apart from the abandon that suffered in previous years, I mean specifically the critical step of getting it through the troubles of the last summer, with the GCC-5/C++11 transition in parallel with a transition of the Boost library (explained in more detail in Work on aptitude). Without that, possibly aptitude would not have survived until today.
  2. Improvements to the suggestions of the resolver In the version 0.8, there were a lot of changes related with improving the order of the suggestions from the resolver, when it finds conflicts or other problems with the planned actions. Historically, but specially in the last few years, there have been many complaints about the nonsensical or dangerous suggestions from the resolver. The first solution offered by the resolver was very often regarded as highly undesirable (for example, removal of many packages), and preferable solutions like upgrades of one or only a handful of packages being offered only after many removals; and keeps only offered as last resort.
Perhaps these changes don't get a lot of attention, given that in the first case it's just to keep working (with few people realising that it could have collapsed on the spot, if left unattended), and the second can probably go unnoticed because it just works or it started to work more smoothly doesn't get as much immediate attention as it suddenly broke! . Still, I wanted to mention them, because I am quite proud of those. Thanks Even if I put a lot of work on aptitude in the last year, the results of the graph and numbers have not been solely achieved by me. Special thanks go to Axel Beckert (abe / XTaran) and the apt team, David Kalnischkies and Julian Andres Klode who, despite the claim in that page, does not mostly work python-apt anymore... but also in the main tools. They help with fixing some of the issues directly, or changing things in apt that benefit aptitude, testing changes, triaging bugs or commenting on them, patiently explaining to me why something in libapt doesn't do what I think it does, and good company in general. Not the least, for holding impromptu BTS group therapy / support meetings, for those cases when prolonged exposure to BTS activity starts to induce very bad feelings. Thanks also to people who sent their translation updates, notified about corrections, sent or tested patches, submitted bugs, or tried to help in other ways. Change logs for details. Notes [1] ^ It's even an example in the Cambridge Dictionaries Online website, for the entry of roller coaster:
He was on an emotional roller coaster for a while when he lost his job.

17 June 2016

Matthias Klumpp: A few words about the future of the Limba project

limba-smallI wanted to write this blogpost since April, and even announced it in two previous posts, but never got to actually write it until now. And with the recent events in Snappy and Flatpak land, I can not defer this post any longer (unless I want to answer the same questions over and over on IRC ^^). As you know, I develop the Limba 3rd-party software installer since 2014 (see this LWN article explaining the project better then I could do  ) which is a spiritual successor to the Listaller project which was in development since roughly 2008. Limba got some competition by Flatpak and Snappy, so it s natural to ask what the projects next steps will be. Meeting with the competition At last FOSDEM and at the GNOME Software sprint this year in April, I met with Alexander Larsson and we discussed the rather unfortunate situation we got into, with Flatpak and Limba being in competition. Both Alex and I have been experimenting with 3rd-party app distribution for quite some time, with me working on Listaller and him working on Glick and Glick2. All these projects never went anywhere. Around the time when I started Limba, fixing design mistakes done with Listaller, Alex started a new attempt at software distribution, this time with sandboxing added to the mix and a new OSTree-based design of the software-distribution mechanism. It wasn t at all clear that XdgApp, later to be renamed to Flatpak, would get huge backing by GNOME and later Red Hat, becoming a very promising candidate for a truly cross-distro software distribution system. The main difference between Limba and Flatpak is that Limba allows modular runtimes, with things like the toolkit, helper libraries and programs being separate modules, which can be updated independently. Flatpak on the other hand, allows just one static runtime and enforces everything that is not in the runtime already to be bundled with the actual application. So, while a Limba bundle might depend on multiple individual other bundles, Flatpak bundles only have one fixed dependency on a runtime. Getting a compromise between those two concepts is not possible, and since the modular vs. static approach in Limba and Flatpak where fundamental, conscious design decisions, merging the projects was also not possible. Alex and I had very productive discussions, and except for the modularity issue, we were pretty much on the same page in every other aspect regarding the sandboxing and app-distribution matters. Sometimes stepping out of the way is the best way to achieve progress So, what to do now? Obviously, I can continue to push Limba forward, but given all the other projects I maintain, this seems to be a waste of resources (Limba eats a lot of my spare time). Now with Flatpak and Snappy being available, I am basically competing with Canonical and Red Hat, who can make much more progress faster then I can do as a single developer. Also, Flatpaks bigger base of contributors compared to Limba is a clear sign which project the community favors more. Furthermore, I started the Listaller and Limba projects to scratch an itch. When being new to Linux, it was very annoying to me to see some applications only being made available in compiled form for one distribution, and sometimes one that I didn t use. Getting software was incredibly hard for me as a newbie, and using the package-manager was also unusual back then (no software center apps existed, only package lists). If you wanted to update one app, you usually needed to update your whole distribution, sometimes even to a development version or rolling-release channel, sacrificing stability. So, if now this issue gets solved by someone else in a good way, there is no point in pushing my solution hard. I developed a tool to solve a problem, and it looks like another tool will fix that issue now before mine does, which is fine, because this longstanding problem will finally be solved. And that s the thing I actually care most about. I still think Limba is the superior solution for app distribution, but it is also the one that is most complex and requires additional work by upstream projects to use it properly. Which is something most projects don t want, and that s completely fine.  And that being said: I think Flatpak is a great project. Alex has much experience in this matter, and the design of Flatpak is sound. It solves many issues 3rd-party app development faces in a pretty elegant way, and I like it very much for that. Also the focus on sandboxing is great, although that part will need more time to become really useful. (Aside from that, working with Alexander is a pleasure, and he really cares about making Flatpak a truly cross-distributional, vendor independent project.) Moving forward So, what I will do now is not to stop Limba development completely, but keep it going as a research project. Maybe one can use Limba bundles to create Flatpak packages more easily. We also discussed having Flatpak launch applications installed by Limba, which would allow both systems to coexist and benefit from each other. Since Limba (unlike Listaller) was also explicitly designed for web-applications, and therefore has a slightly wider scope than Flatpak, this could make sense. In any case though, I will invest much less time in the Limba project. This is good news for all the people out there using the Tanglu Linux distribution, AppStream-metadata-consuming services, PackageKit on Debian, etc. those will receive more attention  An integral part of Limba is a web service called LimbaHub to accept new bundles, do QA on them and publish them in a public repository. I will likely rewrite it to be a service using Flatpak bundles, maybe even supporting Flatpak bundles and Limba bundles side-by-side (and if useful, maybe also support AppImageKit and Snappy). But this project is still on the drawing board. Let s see  P.S: If you come to Debconf in Cape Town, make sure to not miss my talks about AppStream and bundling

Shirish Agarwal: Medical awareness, terrorism, racism and Debconf

Hi to all the souls on planet.debian.org:) . I hope to meet many of you in Debconf16 which is being held at UCT (University of Cape Town), Rhondenbosch, South Africa and am excited to be part of it. I pushed a blog post about my journey to debconf till date. I express my sympathy and condolences to all the people who died in the cowardly shooting spree done by a madman in Orlando . I have been upset about this development but as what s done is done, it s best to just keep moving. Closer to my own reality, I was shocked to discover during my whole visa experience that nowhere there is any knowledge about vaccinations that people should have when they are traveling internationally. I was kinda rudely awakened by this mail which prompted me to take Hep A shot and also start a thread on the mailing list. The more comprehensive info. I got was at the CDC site . It seems to be a go-to site to find about what is recommended. While I will not be taking any shots now as it s nearing to the date of travel, if I had known before, it would have been a valuable resource in itself. Definitely something to be bookmarked if you are going overseas and are worried or have health concerns. On another note, About few days back, there was a discussion as recently, travel advisories have been issued about possible terror action in South Africa by various embassies due to the holy month of Ramadan.
And the recent attack just proves that it takes just one twisted personality with a perverse sense of justice or whatever s/he/ thinks as just to do what s/he did and guns and more security and not the answer.
My take on it some would describe as simplistic, there are things which you can control, there are things beyond your control. No security agency, no country can guarantee it. By being either home or away, you can t wish you will not get bitten. Got images of Final Destination invoked when I was sharing that, for as an Indian, do believe a bit on fate and karma. Also, me being single also plays a part, perhaps I would have been more cautious or have different motivations if I had a family so I do understand some of the concerns which have been raised by people in that thread. At the end of it, it really is a non-choice in my book. If you don t take part due to fear,uncertainty of a possible attack, you have already given in to fear and uncertainty and I believe this goes against the very philosophy of what Debian stands for, being bold, taking chances and having trust in your fellow men. If we haven t allowed proprietary, commercial software to win over us, how can we allow less than 0.1% radically motivated people to scare us ? And the recent attack just proves that it takes just one twisted personality with a perverse sense of justice or whatever s/he/ thinks as just to do what s/he did. More guns and more security are certainly not the answer here. A more troubling part is not terrorism but caste-ism and racism which have been also making news (not in a good way) in India. Now while I cannot claim to have any knowledge about Africans apart from 2-3 conversations which didn t go anywhere, two-three preconceived notions about them can easily be countered. As far as drugs are concerned, IF some Africans are doing drugs smuggling, it would be wrong to pin all of it onto them. India has been fighting drugs smuggling from the 70 s itself. From what we know and have learnt over the years from consuming media, India shares porous borders with almost all our neighbors. Of those, Pakistan is supposed to be the largest grower and supplier and then Nepal where young boys are used as Traffickers. The recent film Rocky Handsome and the recent upcoming controversial movie Udta Punjab are trying to explore these issues. As far as drinking in open is concerned, I have seen and been part of Punjabi parties where both men and women drink without abandon in farm parties. Russians, Germans and Israelis can also out-drink a person on their day/mood. As far as sexuality is concerned, we are the second most populous country in the world, so the less said the better:) . I believe though that underneath this racism is money, greed, phobia, language barriers and just maybe some lifestyle choices as well. I have seen North-Eastern, Chinese, Buddhist people being colored with the same brush. The more interesting case is with the Buddhist as can be seen in Himachal Pradesh, where locals feel they are deprived as Europeans come and indulge Buddhists for their monasteries and their way of life while locals don t get much money from them. This is partly true, but also due to our own short-comings in dealing with westerners. I have never seen my own countrymen going out of their way to make westerners or any tourists for that matter feel welcome in a genuine way. More often, the behavior is between hostility, jealousy and a perverted sense of hero-worship due to the color of skin. As far as being racist and bigoted are concerned, it seems we are not alone, just day before came across this news from Malaysia which is interesting in the sense that how people from different communities frame their own history and people around them, forget that is false and misleading to what we in India know to be the truth, it probably is/would be interesting for somebody who does comparative history, myth and folklore analysis. I was actually planning to talk about some of the talks I am looking forward to hearing and seeing on Debconf but guess that will have to wait for few more days. Hope to publish one more before actually flying to Debconf. Till later.
Filed under: Miscellenous Tagged: #Debconf16, #racism, #terrorism, #vaccinations

15 June 2016

Reproducible builds folks: Reproducible builds: week 59 in Stretch cycle

What happened in the Reproducible Builds effort between June 5th and June 11th 2016: Media coverage Ed Maste gave a talk at BSDCan 2016 on reproducible builds (slides, video). GSoC and Outreachy updates Weekly reports by our participants: Documentation update - Ximin Luo proposed a modification to our SOURCE_DATE_EPOCH spec explaining FORCE_SOURCE_DATE. Some upstream build tools (e.g. TeX, see below) have expressed a desire to control which cases of embedded timestamps should obey SOURCE_DATE_EPOCH. They were not convinced by our arguments on why this is a bad idea, so we agreed on an environment variable FORCE_SOURCE_DATE for them to implement their desired behaviour - named generically, so that at least we can set it centrally. For more details, see the text just linked. However, we strongly urge most build tools not to use this, and instead obey SOURCE_DATE_EPOCH unconditionally in all cases. Toolchain fixes Packages fixed The following 16 packages have become reproducible due to changes in their build-dependencies: apertium-dan-nor apertium-swe-nor asterisk-prompt-fr-armelle blktrace canl-c code-saturne coinor-symphony dsc-statistics frobby libphp-jpgraph paje.app proxycheck pybit spip tircd xbs The following 5 packages are new in Debian and appear to be reproducible so far: golang-github-bowery-prompt golang-github-pkg-errors golang-gopkg-dancannon-gorethink.v2 libtask-kensho-perl sspace The following packages had older versions which were reproducible, and their latest versions are now reproducible again after being fixed: The following packages have become reproducible after being fixed: Some uploads have fixed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Package reviews 68 reviews have been added, 19 have been updated and 28 have been removed in this week. New and updated issues: 26 FTBFS bugs have been reported by Chris Lamb, 1 by Santiago Vila and 1 by Sascha Steinbiss. diffoscope development strip-nondeterminism development disorderfs development tests.reproducible-builds.org Misc. Steven Chamberlain submitted a patch to FreeBSD's makefs to allow reproducible builds of the kfreebsd installer. Ed Maste committed a patch to FreeBSD's binutils to enable determinstic archives by default in GNU ar. Helmut Grohne experimented with cross+native reproductions of dash with some success, using rebootstrap. This week's edition was written by Ximin Luo, Chris Lamb, Holger Levsen, Mattia Rizzolo and reviewed by a bunch of Reproducible builds folks on IRC.

Andrew Shadura: Migrate to systemd without a reboot

Yesterday I was fixing an issue with one of the servers behind kallithea-scm.org: the hook intended to propagage pushes from Our Own Kallithea to Bitbucket stopped working. Until yesterday, that server was using Debian s flavour of System V init and djb s d montools to keep things running. To make the hook asynchronous, I wrote a service to be managed to d montools, so that concurrency issued would be solved by it. However, I didn t implement any timeouts, so when last week wget froze while pulling Weblate s hook, there was nothing to interrupt it, so the hook stopped working since d montools thought it s already running and wouldn t re-trigger it. Killing wget helped, but I decided I need to do something with it to prevent the situation from happening in the future. I ve been using systemd at work for the last year, so I am now confident I m happier with systemd than with d montools, so I decided to switch the server to systemd. Not surprisingly, I prepared unit files in about 5 minutes without having to look into the manuals again, while with d montools I had to check things every time I needed to change something. The tricky thing was the switch itself. It is a virtual server, presumably running in Xen, and I don t have access to the console, so if I bork break something, I need to summon Bradley Kuhn or someone from Conservancy, who s kindly donated the server to the project. In any case, I decided to attempt to upgrade without a reboot, so that I have more options to roll back my changes in the case things go wrong. After studying the manpages of both systemd s init and sysvinit s init, I realised I can install systemd as /sbin/init and ask already running System V init to re-exec. However, systemd s init can t talk to System V init, so before installing systemd I made a backup on it. It s also important to stop all running services (except probably ssh) to make sure systemd doesn t start second instances of each. And then: /tmp/init u and we re running systemd! A couple of additional checks, and it s safe to reboot. Only when I did all that I realised that in the case of systemd not working I d probably not be able to undo my changes if my connection interrupted. So, even though at the end it worked, probably it s not a good idea to perform such manipulations when you don t have an alternative way to connect to the server :)

Russ Allbery: Review: Matter

Review: Matter, by Iain M. Banks
Publisher: Orbit
Copyright: February 2008
ISBN: 0-316-00536-3
Format: Hardcover
Pages: 593
Sursamen is an Arithmetic, Mottled, Disputed, Multiply Inhabited, Multi-million Year Safe, and Godded Shellworld. It's a constructed world with multiple inhabitable levels, each lit by thermonuclear "suns" on tracks, each level supported above the last by giant pillars. Before the recorded history of the current Involved Species, a culture called the Veil created the shellworlds with still-obscure technology for some unknown purpose, and then disappeared. Now, they're inhabited by various transplants and watched over by a hierarchy of mentor and client species. In the case of Sursamen, both the Aultridia and the Oct claim jurisdiction (hence "Disputed"), and are forced into an uneasy truce by the Nariscene, a more sophisticated species that oversees them both. On Sursamen, on level eight to be precise, are the Sarl, a culture with an early industrial level of technology in the middle of a war of conquest to unite their level (and, they hope, the next level down). Their mentors are the Oct, who claim descendance from the mysterious Veil. The Deldeyn, the next level down, are mentored by the Aultridia, a species that evolved from a parasite on Xinthian Tensile Aranothaurs. Since a Xinthian, treated by the Sarl as a god, lives in the heart of Sursamen (hence "Godded"), tensions between the Sarl and the Aultridians run understandably high. The ruler of the Sarl had three sons and a daughter. The oldest was killed by the people he is conquering as Matter starts. The middle son is a womanizer and a fop who, as the book opens, watches a betrayal that he's entirely unprepared to deal with. The youngest is a thoughtful, bookish youth pressed into a position that he also is not well-prepared for. His daughter left the Sarl, and Sursamen itself, fifteen years previously. Now, she's a Special Circumstances agent for the Culture. Matter is the eighth Culture novel, although (like most of the series) there's little need to read the books in any particular order. The introduction to the Culture here is a bit scanty, so you'll have more background and understanding if you've read the previous novels, but it doesn't matter a great deal for the story. Sharp differences in technology levels have turned up in previous Culture novels (although the most notable example is a minor spoiler), but this is the first Culture novel I recall where those technological differences were given a structure. Usually, Culture novels have Special Circumstances meddling in, from their perspective, "inferior" cultures. But Sursamen is not in Culture space or directly the Culture's business. The Involved Species that governs Sursamen space is the Morthanveld: an aquatic species roughly on a technology level with the Culture themselves. The Nariscene are their client species; the Oct and Aultridia are, in turn, client species (well, mostly) of the Nariscene, while meddling with the Sarl and Deldeyn. That part of this book reminded me of Brin's Uplift universe. Banks's Involved Species aren't the obnoxious tyrants of Brin's universe, and mentoring doesn't involve the slavery of the Uplift universe. But some of the politics are a bit similar. And, as with Uplift, all the characters are aware, at least vaguely, of the larger shape of galactic politics. Even the Sarl, who themselves have no more than early industrial technology. When Ferbin flees the betrayal to try to get help, he ascends out of the shellworld to try to get assistance from an Involved species, or perhaps his sister (which turns out to be the same thing). Banks spends some time here, mostly through Ferbin and his servant (who is one of the better characters in this book), trying to imagine what it would be like to live in a society that just invented railroads while being aware of interstellar powers that can do practically anything. The plot, like the world on which it's set, proceeds on multiple levels. There is court intrigue within the Sarl, war on their level and the level below, and Ferbin's search for support and then justice. But the Sarl live in an artifact with some very mysterious places, including the best set piece in the book: an enormous waterfall that's gradually uncovering a lost city on the level below the Sarl, and an archaeological dig that proceeds under the Deldeyn and Sarl alike. Djan Seriy decides to return home when she learns of events in Sarl, originally for reasons of family loyalty and obligation, but she's a bit more in touch with the broader affairs of the galaxy, including the fact that the Oct are acting very strangely. There's something much greater at stake on Sursamen than tedious infighting between non-Involved cultures. As always with Banks, the set pieces and world building are amazing, the scenery is jaw-dropping, and I have some trouble warming to the characters. Dramatic flights across tower-studded landscapes seeking access to forbidden world-spanning towers largely, but don't entirely, make up for not caring about most of the characters for most of the book. This did change, though: although I never particularly warmed to Ferbin, I started to like his younger brother, and I really liked his sister and his servant by the end of the book. Unfortunately, the end of Matter is, if not awful, at least exceedingly abrupt. As is typical of Banks, we get a lot of sense of wonder but not much actual explanation, and the denouement is essentially nonexistent. (There is a coy epilogue hiding after the appendices, but it mostly annoyed me and provides only material for extrapolation about the characters.) Another SF author would have written a book about the Xinthian, the Veil, the purpose of the shellworlds, and the deep history of the galaxy. I should have known going in that Banks isn't that sort of SF author, but it was still frustrating. Still, Banks is an excellent writer and this is a meaty, complex, enjoyable story with some amazing moments of wonder and awe. If you like Culture novels in general, you will like this. If you like set-piece-heavy SF on a grand scale, such as Alastair Reynolds or Kim Stanley Robinson, you probably like this. Recommended. Rating: 8 out of 10

14 June 2016

Simon McVittie: GTK versioning and distributions

Allison Lortie has provoked a lot of comment with her blog post on a new proposal for how GTK is versioned. Here's some more context from the discussion at the GTK hackfest that prompted that proposal: there's actually quite a close analogy in how new Debian versions are developed. The problem we're trying to address here is the two sides of a trade-off: Historically, GTK has aimed to keep compatible within a major version, where major versions are rather far apart (GTK 1 in 1998, GTK 2 in 2002, GTK 3 in 2011, GTK 4 somewhere in the future). Meanwhile, fixing bugs, improving performance and introducing new features sometimes results in major changes behind the scenes. In an ideal world, these behind-the-scenes changes would never break applications; however, the world isn't ideal. (The Debian analogy here is that as much as we aspire to having the upgrade from one stable release to the next not break anything at all, I don't think we've ever achieved that in practice - we still ask users to read the release notes, even though ideally that wouldn't be necessary.) In particular, the perceived cost of doing a proper ABI break (a fully parallel-installable GTK 4) means there's a strong temptation to make changes that don't actually remove or change C symbols, but are clearly an ABI break, in the sense that an application that previously worked and was considered correct no longer works. A prominent recent example is the theming changes in GTK 3.20: the ABI in terms of functions available didn't change, but what happens when you call those functions changed in an incompatible way. This makes GTK hard to rely on for applications outside the GNOME release cycle, which is a problem that needs to be fixed (without stopping development from continuing). The goal of the plan we discussed today is to decouple the latest branch of development, which moves fast and sometimes breaks API, from the API-stable branches, which only get bug fixes. This model should look quite familiar to Debian contributors, because it's a lot like the way we release Debian and Ubuntu. In Debian, at any given time we have a development branch (testing/unstable) - currently "stretch", the future Debian 9. We also have some stable branches, of which the most recent are Debian 8 "jessie" and Debian 7 "wheezy". Different users of Debian have different trade-offs that lead them to choose one or the other of these. Users who value stability and want to avoid unexpected changes, even at a cost in terms of features and fixes for non-critical bugs, choose to use a stable release, preferably the most recent; they only need to change what they run on top of Debian for OS API changes (for instance webapps, local scripts, or the way they interact with the GUI) approximately every 2 years, or perhaps less often than that with the Debian-LTS project supporting non-current stable releases. Meanwhile, users who value the latest versions and are willing to work with a "moving target" as a result choose to use testing/unstable. The GTK analogy here is really quite close. In the new versioning model, library users who value stability over new things would prefer to use a stable-branch, ideally the latest; library users who want the latest features, the latest bug-fixes and the latest new bugs would use the branch that's the current focus of development. In practice we expect that the latter would be mostly GNOME projects. There's been some discussion at the hackfest about how often we'd have a new stable-branch: the fastest rate that's been considered is a stable-branch every 2 years, similar to Ubuntu LTS and Debian, but there's no consensus yet on whether they will be that frequent in practice. How many stable versions of GTK would end up shipped in Debian depends on how rapidly projects move from "old-stable" to "new-stable" upstream, how much those projects' Debian maintainers are willing to patch them to move between branches, and how many versions the release team will tolerate. Once we reach a steady state, I'd hope that we might have 1 or 2 stable-branched versions active at a time, packaged as separate parallel-installable source packages (a lot like how we handle Qt). GTK 2 might well stay around as an additional active version just from historical inertia. The stable versions are intended to be fully parallel-installable, just like the situation with GTK 1.2, GTK 2 and GTK 3 or with the major versions of Qt. For the "current development" version, I'd anticipate that we'd probably only ship one source package, and do ABI transitions for one version active at a time, a lot like how we deal with libgnome-desktop and the evolution-data-server family of libraries. Those versions would have parallel-installable runtime libraries but non-parallel-installable development files, again similar to libgnome-desktop. At the risk of stretching the Debian/Ubuntu analogy too far, the intermediate "current development" GTK releases that would accompany a GNOME release are like Ubuntu's non-LTS suites: they're more up to date than the fully stable releases (Ubuntu LTS, which has a release schedule similar to Debian stable), but less stable and not supported for as long. Hopefully this plan can meet both of its goals: minimize breakage for applications, while not holding back the development of new APIs.

13 June 2016

Reproducible builds folks: First alpha release of reprotest

Author: ceridwen The first, very-alpha release of reprotest is now out at PyPi. It should hit Debian experimental later this week. While it only builds on an existing system (as I'm still working on support for virtualization), it can now check its own reproducibility, which it does in its own tests, both using setuptools and debuild. Unfortunately, setuptools seems to generate file-order-dependent binaries, meaning python setup.py bdist creates unreproducible binaries. With debuild, reprotest probably would be reproducible with the modified packages from the Reproducible Builds project, though I haven't tested that yet. It tests 'captures_environment', 'fileordering' (renamed from 'filesystem'), 'home', 'kernel', 'locales', 'path', 'time', 'timezone', and 'umask'. The other variations require superuser privileges and modifications that would be unsafe to make to a running system, so they will only be enabled in the containers. The next major part of the project is integrating autopkgtests's container management system into reprotest. For the curious, autopkgtest is composed of a main program, adt-run, which then calls other command-line programs, adt-virt-chroot, adt-virt-lxd, adt-virt-schroot, adt-virt-null, adt-virt-schroot, and adt-virt-qemu, that communicate with the containers. (The autopkgtest maintainer has since renamed the programs, but the underlying structure remains the same.) I think this is a bit of an odd design but it works well for my purposes since the container programs already have existing CLIs that reprotest can use.

Next.

Previous.