Search Results: "reg"

16 March 2016

Dirk Eddelbuettel: Rblpapi 0.3.3

A new release of Rblpapi is now on CRAN. It provides a direct interface between R and the Bloomberg Terminal via the C++ API provided by Bloomberg Labs (but note that a valid Bloomberg license and installation is required). This marks the fourth release since the package first appeared on CRAN last year. Thanks to multiple contributors, it packs a number of changes (including two new functions bsrch() and fieldInfo()), numerous internal as well as build improvements, additional documentation and last but not least use of unit testing (provided a Bloomberg connection is available, ie not at CRAN or on Travis CI). See the NEWS.Rd extract for more details:
Changes in Rblpapi version 0.3.3 (2016-03-14)
  • configure adds a missing method="libcurl" options (PR #109 by Martin Bel).
  • New function bsrch() adds basic search functionality (PR #111 by Morgan Williams fixing ticket #82).
  • The licensing status of the Rblpapi source package was clarified (PR #119).
  • The bds() help page now shows an example using an overrides argument (PR #121).
  • A new function fieldInfo was added, and field information is used in bdh and bdp (PRs #122, #123, #125, and #127).
  • The bdp function now checks the order (PR #126).
  • Unit testing was added (PRs #129 and #131).
  • Data retrieval is now smarter about the types returned from Bloomberg (PR #132 and #133; and #141 and #142).
  • The bdh and bds functions now support an option verbose (PRs #138).
Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the Rblpapi page. Questions, comments etc should go to the issue tickets system at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

15 March 2016

Michal Čihař: Weekly phpMyAdmin contributions 2016-W10

Last week was quite equally split between refactoring and bug fixing. As we're getting closer to 4.6.0 release more people give feedback on it and some corner case issues are being discovered. However the biggest challenge was strange bug that was reported long time ago - it affected only Italian translation on Windows server. After lot of debugging, I've realized that trying to encode Cardinalit </td> to json (using json_encode) causes this problem. Obviously there have to be more conditions met as this string only doesn't cause it, however removing it or placing x between and < fixes the problem. In the end I've workarounded it in our code and hopefully I will get to create proper bugreport for php so that it can be fixed upstream as well. The refactoring was mostly focused on encoding conversion part which is now all embedded in the Encoding class. It was quite simplified and should also speed up pages a bit as it does the initialization only when needed not on every phpMyAdmin's page. Handled issues:

Filed under: English phpMyAdmin 0 comments

Dirk Eddelbuettel: x13binary 0.1.2

A new version of x13binary is now on CRAN. It updates the underlying X-13ARIMA-SEATS binary to build 1.1.26 which was released recently by the US Census Bureau. Our x13binary package takes the pain out of installing this, and provides it to the seasonal package upon which other packages such as gunsales or ggseas build their functionality. Things just work: Depend on x13binary and on all relevant OSs supported by R, you should have an X-13ARIMA-SEATS binary installed which will be called seamlessly by the higher-level packages. See this announcement blog post describing the initial 0.1.0 release; we had since followed up with a 0.1.1 release tightening up behaviour on two edge case OSs. Courtesy of CRANberries, there is also a diffstat report for the most recent release as well as the preceding bugfix release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

14 March 2016

Julian Andres Klode: Dropping SHA-1 support in APT

Tomorrow is the anniversary of Caesar s assassination APT will see a new release, turning of support for SHA-1 checksums in Debian unstable and in Ubuntu xenial, the upcoming LTS release. While I have no knowledge of an imminent attack on our use of SHA1, Xenial (Ubuntu 16.04 LTS) will be supported for 5 years, and the landscape may change a lot in the next 5 years. As disabling the SHA1 support requires a bit of patching in our test suite, it s best to do that now rather than later when we re forced to do it. This will mean that starting tomorrow, some third party repositories may stop working, such as the one for the web browser I am writing this with. Debian Derivatives should be mostly safe for that change, if they are registered in the Consensus, as that has checks for that. This is a bit unfortunate, but we have no real choice: Technical restrictions prevent us from just showing a warning in a sensible way. There is one caveat, however: GPG signatures may still use SHA1. While I have prepared the needed code to reject SHA1-based signatures in APT, a lot of third party repositories still ship Release files signed with signatures using SHA-1 as the digest algorithms. Some repositories even still use 1024-bit DSA keys. I plan to enforce SHA2 for GPG signatures some time after the release of xenial, and definitely for Ubuntu 16.10, so around June-August (possibly during DebConf). For xenial, I plan to have a SRU (stable release update) in January to do the same (it s just adding one member to an array). This should give 3rd party providers a reasonable time frame to migrate to a new digest algorithm for their GPG config and possibly a new repository key. Summary
Filed under: Debian, Ubuntu

13 March 2016

Ingo Juergensmann: Letsencrypt: challenging challenges solved

A few weeks ago I was wondering in Letsencrypt: challenging challenges about how to setup Letsencrypt when a domain is spread across several virtual machines (VM). One of the possible solutions would be to consolidate everything on one single VM, which is nothing I would like to do. The second option would need to generate the Letsencrypt certs on the webserver and copy over the certs to the appropriate VM on a regular basis or event driven. The third option is to use a network share - and this is what I'm using right now. So, my setup is as following after I solved the GlusterFS issue with rpcbind binding to all interfaces, although it has been configured to only listen to certain interfaces (solution was: simply remove all NFS related stuff): On Dom0 (or the host machine) I run GlusterFS as a server on a small 1 GB LVM as part of a replicate with the VM that will do the actual Letsencrypt work:
Volume Name: le
Type: Replicate
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: 192.168.x.254:/srv/gfs/le
Brick2: 192.168.x.1:/srv/gfs/le
This is to ensure that on reboot of the machine every other VM using Letsencrypt certs can mount the GlusterFS share, because the host machine will be there for sure whereas the other VM generating the certs with the letsencrypt.sh script might still be booting. And when the GlusterFS share is missing services will not start on the other VMs because of the missing certs, of course. So, the replica on the virtualization host (Dom0) is only acting as some kind of always-being-available network share, because, well, the other VM will not always be there... for example during a kernel update when a reboot is required. The same setup is on my mailserver, acting as the second GlusterFS brick of that replica drive. The mailserver hosts the bind9 nameserver as well and I might do something that new domains with Letsencrypt certs get added to my DNSSEC setup as well. Of course, when the letsencrypt.sh script creates or updates the certs, it needs the certs being mounted in that configured location, so I needed to add a line to /etc/fstab:
192.168.x.254:/le /etc/letsencrypt.sh/certs glusterfs noexec,nodev,_netdev 0 0
Basically the same needs to be done on the other VMs where you want to use the certs as well, but you may want to mount the share as read-only there. The next step was a little more tricky. When letsencrypt.sh generates new certs, Letsencrypt will contact the webserver for that domain to respond to the ACME challenge. This requires that on each VM you want to use letsencrypt you have to run a webserver. Well, actually at least that there is somewhere a webserver that can answer these requests for that specific domain... Now, the setup of the webserver (Apache in my case) is like this: I'm using the Apache macro module to make it more easy, so I generated two small configs in /etc/apache/conf-available and enabled them bei a2enconf: letsencrypt-proxy.conf to do some setup for proxying the ACME challenges to a common website called acme.example.org. And then letsencrypt-sslredir.conf to setup SSL redirection when everything is in place and the domain can be switched over to HTTPS-only. letsencrypt-proxy.conf:
<Macro le_proxy>
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass /.well-known/acme-challenge/ http://acme.windfluechter.net/
ProxyPassReverse / http://% HTTP_HOST /.well-known/acme-challenge/
</Macro>
letsencrypt-sslredir.conf:
<Macro le_sslredir>
RewriteEngine on
RewriteCond % HTTPS !=on
RewriteRule . https://% HTTP_HOST % REQUEST_URI [L]
</Macro>
So, after all the setup of a virtual host for Apache looks like this:
<Macro example.org>
(lots of setup stuff)
</Macro>
<VirtualHost 31.172.31.x:443 [2a01:a700:4629:x::1]:443>
Header always set Strict-Transport-Security "max-age=31556926; includeSubDomains"
SSLEngine on
# letsencrypt certs:
SSLCertificateFile /etc/letsencrypt.sh/certs/example.org/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt.sh/certs/example.org/privkey.pem
SSLHonorCipherOrder On
Use example.org
Use le_proxy
</VirtualHost>
<VirtualHost 31.172.31.x:80 [2a01:a700:4629:x::1]:80>
Use example.org
Use le_proxy
Use le_sslredir
</VirtualHost>
le_sslredir is only needed when you are sure that you want all traffic being redirected to HTTPS. For example when your blog is listed on planet.debian.org or other Planets you might want to omit this from your HTTP config because bug #813313 is not yet solved. In the end, when creating a new Letsencrypt cert, you need to add the le_proxy macro to your website, add the domain to letsencrypt.sh config in /etc/letsencrypt.sh/domains.txt and then the scripts will request a new cert from Letsencrypt, handling the ACME challenge stuff via the URL redirection in le_proxy being redirected to your acme.exmaple.org site and finally writes your new cert to the GlusterFS share. From that share you can then use the new cert on all needed VMs, be it your mailserver, webserver or XMPP/SIP server VMs. At least this works for me.
Kategorie:

11 March 2016

Steinar H. Gunderson: Agon and the Candidates tournament

The situation where Agon (the designated organizer of the Chess World Championship, and also the Candidates tournament, the prequalifier to said WC) is trying to claim exclusive rights of the broadcasting of the moves (not just the video) is turning bizarre. First of all, they have readily acknowledged they have no basis in copyright to do so; chess moves, once played, are facts and cannot be limited. They try to jump through some hoops with a New York-specific doctrine (even though the Candidates, unlike the World Championship, is played in Moscow) about hot news , but their main weapon seems to be that they simply will throw out anyone from the hall who tries to report on the moves, and then try to give them only to those that promise not to give them on. This leads to the previously unheard-of situation where you need to register and accept their terms just to get to watch the games in your browser. You have to wonder what they will be doing about the World Championship, which is broadcast unencrypted on Norwegian television (previous editions also with no geoblock). Needless to say, this wasn't practically possible to hold together. All the big sites (like Chessdom, ChessBomb and Chess24) had coverage as if nothing had happened. Move sourcing is a bit of a murky business where nobody really wants to say where they get the moves from (although it's pretty clear that for many tournaments, the tournament organizers will simply come to one or more of the big players with an URL they can poll at will, containing the games in the standard PGN format), and this was no exception ChessBomb went to the unusual move of asking their viewers to download Tor and crowdsource the moves, while Chessdom and Chess24 appeared to do no such thing. In fact, unlike Chessdom and ChessBomb, Chess24 didn't seem to say a thing about the controversy, possibly because they now found themselves on the other side of the fence from Norway Chess 2015, where they themselves had exclusive rights to the PGN in a similar controversy although it would seem from a tweet that they were perfectly okay with people just re-broadcasting from their site if they paid for a (quite expensive) premium membership, and didn't come up with any similar legal acrobatics to try to scare other sites. However, their ToS were less clear on the issue, and they didn't respond to requests for clarification at the time, so I guess all of this just continues to be on some sort of gentleman's agreement among the bigger players. (ChessBomb also provides PGNs for premium members for the tournaments they serve, but they expressly prohibit rebroadcast. They claim that for the tournaments they host, which is a small minority, they provide free PGNs for all.) Agon, predictably, sent out angry letters where they threatened to sue the sites in question, although it's not clear at all to me what exactly they would sue for. Nobody seemed to care, except one entity TWIC, which normally has live PGNs from most tournaments, announced they would not be broadcasting from the Candidats tournament. This isn't that unexpected, as TWIC (which is pretty much a one-man project anyway) mainly is about archival, where they publish weekly dumps of all top-level games played that week. This didn't affect a lot of sites, though, as TWIC's live PGNs are often not what you'd want to base a top-caliber site on (they usually lack clock information, and moves are often delayed by half a minute or so). I run a hobby chess relay/analysis site myself (mainly focusing on the games of Magnus Carlsen), though, so I've used TWIC a fair bit in the past, and if I were to cover the Candidates tournament (I don't plan to do so, given Agon's behavior, although I plan to cover the World Championship itself), I might have been hit by this. So, that was the background. The strange part started when worldchess.com, Agon's broadcasting site, promptly went down during the first round of the Candidates tournament today Agon blamed DDoS, which I'm sure is true, but it's unclear exactly how strong the DDoS was, and if they did anything at all to deal with it other than to simply wait it out. But this lead to the crazy situation where the self-declared monopolist was the only big player not broadcasting the tournament in some form. And now, in the trully bizarre move, World Chess is publishing a detailed rebuttal of Agon's arguments, explaining how it is bad for chess, not juridically sound, and also morally wrong. Yes, you read that right; Agon's broadcast site is carrying an op-ed saying Agon is wrong. You at least have to give them credit for not trying to censor their columinst when he says something they don't agree with. Oh, and if you want those PGNs? I will, at least for the time being, be pushing them out live on http://pgn.sesse.net/. I have not gone into any agreement with Agon, and they're hosted in Norway, far from any New York-specific doctrines. So feel free to relay from them, although I would of course be happy to know if you do.

Alessio Treglia: We need creativity: which one?

A constant demand for creativity is raising from every corner of the Western world, from any business sector or professional activity, by individual or communities. This term is used everywhere, even in advertising to attract the attention of consumers: as a thirsty wanderer lost in the desert sand, the need for creativity seems to be the source of an oasis of salvation. albaJulien Ries anthropological research showed us that, already more than two million years ago, Homo Habilis looks like Symbolicus, with aesthetic sensibility, sense of symmetry and consciousness of creativity. Gilbert Durand confirms that the specific activity of man, the identity card of Homo Sapiens, is the symbolic activity, an essential part of his creativity. Then, man is creative at the moment when his first activates his imaginative feature. So we can ask ourselves, how did we miss the creativity of man, of which so much we feel the need, or at least where is it hiding now? But above all which kind of creativity are we talking about? <Read More >

Dirk Eddelbuettel: Rcpp now used by 600 CRAN packages

600 Rcpp packages Earlier this morning, Rcpp reached another milestone: 600 packages on CRAN now depend on it (as measured by Depends, Imports and LinkingTo declarations). The graph is on the left depicts the growth of Rcpp usage over time. Rcpp cleared 300 packages in November 2014. It passed 400 packages in June of last year (when I only tweeted about it) and then 500 packages in late October. The chart now extends to the very beginning as I manually compiled some more data from the CRANberries checked with crandb. Then next part uses manually saved entries, and the final and largest part of the data set was generated semi-automatically via a short script appending updates to a small file-based backend. A list of user package is kept on this page. Also displayed in the graph is the relative proportion of CRAN packages using Rcpp. The four per-cent hurdle was cleared just before useR! 2014 where I showed a similar graph (as two distinct graphs) in my invited talk. We passed five percent in December of 2014, six percent last July, seven percent just before Christmas and now stands at 7.31 percent, or a little less than one in thirteen R packages. 600 user packages is truly humbling, a staggering number and a big responsibility. We continue to try our best try to keep Rcpp as performant and reliable as it has been so that the next set of packages can rely on it---just like these 600 do. So with that a very big Thank You! to all users and contributors of Rcpp for help, suggestions, bug reports, documentation or, of course, code.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

10 March 2016

Lunar: Reproducible builds: week 45 in Stretch cycle

What happened in the reproducible builds effort between February 28th and March 5th:

Toolchain fixes
  • Antonio Terceiro uploaded gem2deb/0.27 that forces generated gemspecs to use the date from debian/changelog.
  • Antonio Terceiro uploaded gem2deb/0.28 that forces generated gemspecs to have their contains file lists sorted.
  • Robert Luberda uploaded ispell/3.4.00-5 which make builds of hashes reproducible.
  • C dric Boutillier uploaded ruby-ronn/0.7.3-4 which will make the output locale agnostic. Original patch by Chris Lamb.
  • Markus Koschany uploaded spring/101.0+dfsg-1. Fixed by Alexandre Detiste.
Ximin Luo resubmitted the patch adding the --clamp-mtime option to Tar on Savannah's bug tracker. Lunar rebased our experimental dpkg on top of the current master branch. Changes in the test infrastructure are required before uploading a new version to our experimental repository. Reiner Herrmann rebased our custom texlive-bin against the latest uploaded version.

Packages fixed The following 77 packages have become reproducible due to changes in their build dependencies: asciidoctor, atig, fuel-astute, jekyll, libphone-ui-shr, linkchecker, maven-plugin-testing, node-iscroll, origami-pdf, plexus-digest, pry, python-avro, python-odf, rails, ruby-actionpack-xml-parser, ruby-active-model-serializers, ruby-activerecord-session-store, ruby-api-pagination, ruby-babosa, ruby-carrierwave, ruby-classifier-reborn, ruby-compass, ruby-concurrent, ruby-configurate, ruby-crack, ruby-css-parser, ruby-cucumber-rails, ruby-delorean, ruby-encryptor, ruby-fakeweb, ruby-flexmock, ruby-fog-vsphere, ruby-gemojione, ruby-git, ruby-grack, ruby-htmlentities, ruby-jekyll-feed, ruby-json-schema, ruby-listen, ruby-markerb, ruby-mathml, ruby-mini-magick, ruby-net-telnet, ruby-omniauth-azure-oauth2, ruby-omniauth-saml, ruby-org, ruby-origin, ruby-prawn, ruby-pygments.rb, ruby-raemon, ruby-rails-deprecated-sanitizer, ruby-raindrops, ruby-rbpdf, ruby-rbvmomi, ruby-recaptcha, ruby-ref, ruby-responders, ruby-rjb, ruby-rspec-rails, ruby-rspec, ruby-rufus-scheduler, ruby-sass-rails, ruby-sass, ruby-sentry-raven, ruby-sequel-pg, ruby-sequel, ruby-settingslogic, ruby-shoulda-matchers, ruby-slack-notifier, ruby-symboltable, ruby-timers, ruby-zip, ticgit, tmuxinator, vagrant, wagon, yard. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet:
  • #816209 on elog by Reiner Herrmann: use printf instead of echo which is shell-independent.
  • #816214 on python-pip by Reiner Herrmann: removes timestamp from generated Python scripts.
  • #816230 on rows by Reiner Herrmann: tell grep to always treat the input as text.
  • #816232 on eficas by Reiner Herrmann: use printf instead of echo which is shell-independent.
Florent Daigniere and bancfc reported that linux-grsec was currently built with GRKERNSEC_RANDSTRUCT which will prevent reproducible builds with the current packaging.

tests.reproducible-builds.org pbuilder has been updated to the last version to be able to support Build-Depends-Arch and Build-Conflicts-Arch. (Mattia Rizzolo, h01ger) New package sets have been added for Subgraph OS, which is based on Debian Stretch: packages and build dependencies. (h01ger) Two new armhf build nodes have been added (thanks Vagrant Cascadian) and integrated in our Jenkins setup with 8 new armhf builder jobs. (h01ger)

strip-nondeterminism development strip-nondeterminism version 0.016-1 was released on Sunday 28th. It will now normalize the POT-Creation-Date field in GNU Gettext .mo files. (Reiner Herrmann) Several improvements to the packages metadata have also been made. (h01ger, Ben Finney)

Package reviews 185 reviews have been removed, 91 added and 33 updated in the previous week. New issue: fileorder_in_gemspec_files_list. 43 FTBFS bugs were reported by Chris Lamb, Martin Michlmayr, and gregor herrmann.

Misc. After merging the patch from Dhiru Kholia adding support for SOURCE_DATE_EPOCH in rpm, Florian Festi opened a discussion on the rpm-ecosystem mailing list about reproducible builds. On March 4th, Lunar gave an overview of the general reproducible builds effort at the Internet Freedom Festival in Valencia.

Antoine Beaupr : Trying out Keybase

I was privileged enough to get access to the alpha preview of Keybase.io. Last year, the project raised 11M$ in venture capital and they seem to be attacking the hard problems, so it is a serious project, worth considering for any cryptography hacker like me. This was obviously discussed elsewhere previously but since Keybase has evolved a bit since then, I figured it would be worth adding my grain of salt.

The elite alpha problem My first problem with Keybase was of course that I wasn't in the elite club of invited people. Somehow, I finally got in but the problem remains for the everyone else. In my case, it was the Keybase filesystem announcement that brought my attention back to the project, although I have yet to test that itself - yet another invite-only problem (need to email chris@keybase.io to get access, I guess). Still, 10GB of storage might be interesting for some people, but considering the number of hurdles or "friction", as they say, that needs to be jumped over to get access, I am not sure how much traction this will get for now. The project has been in alpha for a while now (more than a year) with no announced public beta just yet, which is way slower than similarly challenging projects with less capital. Let's encrypt, as a comparison, was publicly announced in 2014 and by the end of 2015, the public beta was opened.

Private key leakage My second major concern is that Keybase does worrisome things to private key material. If you tell it to import your regular GPG keys, it will copy them into the keybase keyring, which is already a problem: now you have two places where you store sensitive secrets. But worse, Keybase offers you to upload your private key material on the server, without clearly and explicitely exposing the potential risks that means to the user. The key, presumably, is encrypted (at least that is what I can parse from the Keybase privacy policy). Yet this is still a concern: a determined attacker (e.g. the government or another malicious entity) could force Keybase to give up the encrypted private key material and install sniffers that would reveal the passphrase that would allow decryption and impersonation of the related identity. That is a huge deal. This is what caused Hushmail's demise in 2007: they had similar promises about how they had a "secure" email (in this case) service based on OpenPGP, but since they key was stored on the server, they only had to serve a malicious applet to victims and were able to give cleartext data to the US government (12 CDs!) under a Mutual Legal Assistance Treaty with Canada. In Keybase, When I generate a new key, it asks me if I want to upload the key to their servers, and "Yes" is chosen as a default, so it is clearly a policy that is encouraged. This is presumably to enable browser-based crypto, yet this could be done by storing keys within the browser, not on the server. The mere possibility that the client can meddle with secret key material in such a way is a problem. Even if uploading secret keys to the server is optional, the fact that it is a possibility widens the attack surface on the client significantly. GPG has been working on isolating private key operations to a different process (gpg-agent) or smart cards, SSH has privilege seperation. The idea of uploading secret key material online is so beyond current practices that it should bring the project to a full stop already. My private key material is pretty sensitive. I am a Debian developer: if someone gets access to that key, they can upload arbitrary code for thousands of packages, affecting thousands if not millions of users. That is a huge deal, and I am not ready to give that power to an arbitrary third party that I know little about.

Privacy policy issues Furthermore, the Keybase privacy policy explicitely states that "Logs of this information may persist for an indefinite period", which include, and I quote:
  • Your computer s IP address
  • Your preferences and settings (time zone, language, privacy preferences, application preferences, etc.)
  • The URL of the site that referred you to the Service
  • The buttons and controls you clicked on (if any) within the Service
  • How long you used the Service and which parts and features you used
  • Session times and lengths
For a company working to improve user's privacy, this is a pretty bad policy. It exposes Keybase users to undue surveillance, on a lot of private information that is not necessary to operate the service. I haven't actually read the remaining of the policy, as I was already scared as hell and figure I would just log off for now and try not to add any extra personal information in that already growing pile. Let's mention, while I'm here, that the keybase commandline client has this odd way of working that it forks a daemon in the background all the time, even to run just keybase status. One has to wonder if this thing is included in the surveillance data above, which would make the keybase client an amazing surveillance device in itself. To stop all this, you need:
keybase logout
keybase ctl stop
Not quite obvious...

Usability issues Which brings me to a series of usability issues I have found while working with Keybase. Having worked on usability myself in the Monkeysphere project, I understand how hard those problems can be. But right now, the fact is that keybase is only a commandline client, with some web-based sugar sprinkled on top. I recognize the huge efforts that have been done to make the user experience (UX) as easy as possible through the commandline, but this is definitely not "grand public" material yet. Even as a OpenPGP hacker, Keybase can get confusing. Since it has its own separate keyring from GPG, things that used to be obvious before are suddenly new things to learn. For example, I originally generated a key for anarcat@debian.org in keybase, thinking it would be a separate thing from my current GPG identity. Well, that ended up creating a new key in my GPG keyring, duplicating my existing identity, with no way for me to tell this was a Keybase copy. This could have ended up on the keyservers and confuse anyone looking for my PGP key.

Revoking keys So I revoked that key, which, in itself, is not such an obvious process either. I first had to "drop" the key:
keybase pgp drop '010143572d4e438f457a7447c9758804cc7be44c1ee2b7915c3904567d0a3fb5cf590a'
To find that magic number, I had to go on the website and click through the PGP key to end up on the revoke link which told me which magic string to put. The web UI was telling me I could use keybase status to get that information, but I couldn't find that in the output here:
$ keybase status
Username:      anarcat
Logged in:     no
Device:
    name:      angela
    ID:        070f6a13f2a66afbb463f49dadfd4518
    status:    active
Keybase keys:  unlocked
Session:       anarcat [salt only]
KBFS:
    status:    not running
    version:
    log:       /home/anarcat/.cache/keybase/keybase.kbfs.log
Service:
    status:    running
    version:   1.0.14-1
    log:       /home/anarcat/.cache/keybase/keybase.service.log
Platform Information:
    OS:        linux
    Runtime:   go1.5.3
    Arch:      amd64
Client:
    version:   1.0.14-1
Desktop app:
    status:    not running
    version:
    log:       /home/anarcat/.cache/keybase/Keybase.app.log
Config path:   /home/anarcat/.config/keybase/config.json
Default user:  anarcat
Other users:
command-line client:
    keybase status [pid: 14562, version: 1.0.14-1]
So keybase drop "deletes" the key from the keybase client (and presumably the server). One problem here is that we have three different verbs to mean the same thing: in some places, the website says "deleted" (in the web graph), the commandline client uses "drop" and everyone else using PGP in the world uses "revoke". So what is it? Also note that just dropping that key is not enough to eradicate that key: you still need to generate and import a revocation certificate on GPG's side as well, for that key. This makes sense, because you may have imported that key and you may not want to destroy it when you remove it from keybase, but there could be a little better guessing and UX here as well.

Password usage Then another problem is that, when you signup on the commandline, it asks you to choose a passphrase. It's not clear at all from the UI what that passphrase is for. Is it to protect the private key material? To login the website? Both? It turns out it is actually to login to the website and the Keybase API. It could also be the key to the private key material, but I haven't quite figured that out yet. Similarly, the "paper devices" are a little confusing to me as well. The registration process is fairly insistent that I need to write a series of secret words down on a piece of paper and put it my wallet. This is based on the idea that you can use that key to recover your account and setup new devices. However, it is not made clear at all how those credentials should be protected or used. What if I lose my wallet? What do I do then? Is it okay if i write it on the back of my laptop instead? I know it sounds like stupid questions to crypto geek, but it's definitely stuff people will do, and education, at all levels, is necessary here.

The existing web of trust Finally, I found it surprisingly counter-intuitive to sign keys with keybase. I was assuming the whole point of the thing was to expand the web of trust, but it seems their mission lies somewhere else: there is no facility to sign keys directly in keybase. You sign statements on various social sites (Github, Twitter, Reddit...) but doing traditional key exchanges seems to be off limits. My obvious use case was to sign my existing key material with my newly generated anarcat@keybase.io key, so that keybase would become another way of verifying my identity. Well, this is not possible with keybase directly - you still have to go through the regular:
gpg --sign-key DEADBEEF
Which is really unfortunate, because some craaaazy people like me insist on doing in person, actually secure key exchanges sometimes, and the tools to do this right now (mainly gpg, caff and monkeysign) really need a lot of love and help. Keybase could have helped a lot in pushing those initiatives forward and bring new ways of doing secure key exchanges, instead of only lowering the standards on cryptographic identity.

Usual proprietary startup rant I can't help but finish by noticing that the whole basic problem with Keybase is more fundamental than a few usability issues, which can all be fixed fairly easily. Keybase.IO is a web-based service that is entirely proprietary. The client is freely downloadable and free software - but the server side is not. This makes us rely on a central point of failure and the goodwill of those operators to not only keep the service running (for free?) indefinitely, but to protect us against all attackers, state-run or otherwise. This is definitely against basic free software principles and the open architecture of the internet federation, well embodied in the Franklin street statement. Online services should be decentralized and open like the OpenPGP keyservers have always been, period.

Conclusion I am curious to see where Keybase brings us, as a community. So far, I am concerned about centralization and devaluation of the web of trust as cryptographic system. I do not believe that trusting multiple corporate social networking sites bring much benefit to our security, although it does improve usability significantly and is certainly better than TOFU, so that part of the project is definitely interesting, especially since it allows leveraging classical internet protocols like DNS and HTTPS. Key exchange is a critical problem that still hasn't properly been resolved in the cryptographic community. Most efforts at building communication tools (say like Signal or Telegram) mostly ignore the problem and expect people to read up strings of numbers or rely on synchronous communications (see the Axolotl rachet for Signal's take on it). Keybase's attempts at fixing this are great, but needs much more work to actually resolve the PKI issues in a significant way. More fundamentally, the practice of storing keys on the servers should just stop: it is a definite no go for me, and a classic crypto mistake that has bitten way too many people. In my mind, it discredits the whole project. The point of OpenPGP is end to end encryption across devices: storing the keys on a server breaks that apart, and doesn't improve much on the existing HTTPS security systems already in place on the web. It is especially a problem given the new waves of attacks on cryptography from western governments, from the UK to California... Until those issues are resolved, I can't get myself to recommend Keybase to anyone just yet.

Post-scriptum Note to Keybase developers: I have considered filing issues regarding the above, but unfortunately, I was unable to filter through the gigantic github issues list for duplicates, and didn't have time to file detailed reports for everything. I apologize for bypassing those usual conventions. If you want to try out Keybase to make your own opinion, I do have 4 invites I can give away, but I will do so only if you need to test specific issues. Otherwise, I note that there is a Request For Package for the Debian package to become official, that Debian developers (or Keybase developers!) may be interested in completing.

Michal &#268;iha&#345;: Weblate 2.5

After almost six months of development Weblate 2.5 has been released. It brings lot of improvements and it's quite hard to point few ones. The most important ones include support for Python 3, reports generators, placeables highlighting, extended keyboard shortcuts, configurable dashboard or group based ACLs. Full list of changes for 2.5: If you are upgrading from older version, please follow our upgrading instructions. You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects. Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure. Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

Filed under: English Weblate 0 comments

Hideki Yamane: Microsoft uses Debian - this is 2010s

Interesting, the Register says Microsoft has released a Debian Linux switch OS. Repeat, a Debian Linux switch operating system, I don't know it's true or not but interesting at least.

Anyway, relationship between Debian and Microsoft is completely different from 10 years ago. Microsoft Azure added Debian as one of their endorsed distribution, and I worked for improving Hyper-V support on Debian, it becomes better than until 8.2 with hyperv-daemons package (thanks to folks for pushing it to stable). It's good for everyone that one of the biggest software company chose Debian for their product.


And, dear Microsoft, if you truely love OSS, then it's time to consider to show it with becoming sponsor for DebConf and talk with us in there with beer :)

swirl, swirl

9 March 2016

Joerg Jaspert: Squeeze fully archived

While writing this, the last bits of squeeze that haven t been on archive.debian.org yet are moved over there, that is security and the lts suite. As soon as that script done, some tiny archive magic will happen and over the next few dinstall runs, the rest of squeeze that still was there will no longer be on our regular mirror network anymore. If you still use it, now is the time to upgrade. Update: Forgot to mention, but yes, squeeze-backports(-sloppy) is also archived on archive.debian.org.

8 March 2016

Dirk Eddelbuettel: gtrendsR 1.3.3

A very nice new update to the gtrendsR package by Philippe and myself is now avilable via CRAN. I had only blogged about the initial 1.3.0 release, and we have added a whole slew of new features and fixes. Philippe rewrote a lot of the parsing to make it more robust to different encodings, and to add other features. So in no particular order, we can now sub-group by regions more finely, withstand various misfeatures in returned data sets, generally do better on connections, and more --- and also allow for intra-day, daily and weekly queries! That last part is pretty fun. Here is the code I ran last Saturday to look at the query for Donald Drumpf, a name brought to us via a beautiful John Oliver episode worth watching which ran about nine days ago. So last Saturday, when we were still within the seven day window, I ran
library(gtrendsR)
dp <- gtrends("Donald Drumpf", res="7d")
plot(dp) + ggplot2::ggtitle("The Drumpf") + ggplot2::theme(legend.position="none")
which resulted in the following chart Donald Drump query which highlights another nice feature: the ggplot2 object created by the plotting function is returned, so we can locally modify and tune it. Here we set a title and suppress the default legend. As I had not blogged about the interim bug-fix releases 1.3.1 and 1.3.2, here is the set of NEWS entries for the last three releases:

gtrendsR 1.3.3
  • A ggplot2 object can now be returned for further customization. plot(gtrends("NHL")) + ggtitle("NHL trend") + theme(legend.position="none")
  • Support for hourly and daily data (#67). For example, it is now possible to have hourly data for the last seven days with gtrends("nhl", geo = "CA", res = "7d"). Use ?gtrends for more information about the time resolution supported by the package.
  • Support for categorties (#46). Ex.: gtrends("NHL", geo = "US", cat = "0-20") will search only in the sport category.
  • Some countries (ex: Hong Kong) were missing from the list (#69).
  • Various typos and documentation work.

gtrendsR 1.3.2
  • Added support for sub-countries (#25). Ex.: gtrends("NHL", geo = "CA-QC") will return trends data for Qu bec province in Canada. The list of supported sub-countries can be obtained via data(countries).
  • Data parsing should work for any data returned by Google Trends (i.e. countries independent).
  • Better support for queries using keywords in different languages (#50, #57). Ex.: gtrends(" ", geo = "TW")
  • Now able to specify up to five countries (#53) via gtrends("NHL", geo = c("CA", "US"))
  • Fixing issue #51 allowing UK-based queries via geo = "GB"

gtrendsR 1.3.1
  • Fixing issue #34 where connection verification was not done properly.
  • Now able to use more latin character in query. For example: gtrends("montr al").
  • Can now deal with data returned other than in English language.

Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the gtrendsR repo where questions, comments etc should go via the issue tickets system.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Edited 2016-03-08: Corrected code snipped and one grammar instance

7 March 2016

Laura Arjona Reina: More involved in the Debian Publicity team

More than 6months since I am Debian Developer and I m learning new things every day and trying to organize my time better to accomplish all the things I would like to do.
In September I became publicity delegate (and we joined Press+Publicity into the Publicity team). We continued discussing how the team works, and hopefully revitalizing it.
I became more active in the IRC channel, installed quassel in my home server and then I could read all the backlog too.
I ve tried to contribute more to Debian Project News (less than I d like), to bits.debian.org and to the social networks (Pump.io and GNU Social).
Since November I ve been failing to call for meeting but finally on 2016/02/29 we had it. I hope to resume semi-regular meetings each 1-2 months.
At the beginning of the year I helped to handle all the news and announcements about the loss of Ian Murdock. It has been hard work but I feel that I could turn some of the sadness into something useful for others. There is still work to be done: parse the condolence emails and setup a website to publish them.
About bits.debian.org, DPN and announcements, I ve learned how to perform the parts of the work that require membership permissions (building the blog, actually publishing the announcement, sending the DPN/announcement mail to the corresponding lists). Not without mistakes, but I guess no matter how much you care, sometimes things happen, and then you learn and then things get better and you too.
Some things I discovered Some clarifications
  • I ve written hard word 4 times in this post (well, 5 now!) but that does not mean I don t want to do those tasks. I feel going out of my comfort zone and that s ok, needed to learn and experience. I m having a good time in Debian in the last months, as always!
  • We re 4 people delegated and more team members and contributors, so anybody could think: Publicity is well covered, let s go to do other things in Debian . It s perfectly ok if you want to do other things in Debian, but please consider combining your contributions with some minutes for the publicity team. Aiming to be the universal operating system, our community and target audience is big and diverse and we d like to show that diversity to the world. The more we are, the merrier!
Comments?
You can comment about this blog post in this pump.io thread.

Filed under: My experiences and opinion Tagged: Communities, Contributing to libre software, Debian, Developer motivations, English, Free Software

Dirk Eddelbuettel: Rblpapi 'Release Candidate' 0.3.2.5

We have a made numerous changes to Rblpapi since the previous release 0.3.2 in early December. You can see the commits, or look at the ChangeLog or NEWS.Rd to get an indea of the changes. We have new functions, improved internals, bug fixes and more. In order to facilitate more widespread testing, I have just placed source and (Windows) binaries in the ghrr drat repo from which you can install the new version simply via
drat:::add("ghrr")          # if you have drat installed
install.packages("Rblpapi") # uses the drat version b/c higher version number than release
Alternatively you can also do
drat:::add("ghrr")          # if you have drat installed
update.packages()           # refresh all packages against all repos
We would appreciate wider testing, and feedback / bug reports / ... via the issue tracker. PRs with unit test suggestions would also be most welcome -- we now use RUnit and run the tests if a file ~/.R/rblpapiOptions.R exists which sets the options() values for automatic connection (see help(blpConnect)) as well as the blpUnitTests=TRUE option. This is needed to 'opt-in' as standard test setups at [Travis])(https://travis-ci.org/) or CRAN will not have access to a Bloomberg terminal.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

6 March 2016

Thorsten Glaser: mksh R52c, paxmirabilis 20160306 released; PA4 paper size PDF manpages

The MirBSD Korn Shell R52c was published today as bugfix-accumulating release of low upto medium importance. Thanks to everyone who helped squashing all those bugs; this includes our bug reporters who always include reproducer testcases; you re wonderful! MirCPIO was also resynchronised from OpenBSD, to address the CVE-2015- 1193,1194 test cases, after a downstream (wow there are so many?) reminded us of it; thanks!
This is mostly to prevent extracting ../foo either directly or from a symlink(7) from actually ending up being placed in the parent directory. As such the severity is medium-high. And it has a page now initially just a landing page / stub; will be fleshed out later. Uploads for both should make their way into Debian very soon (these are the packages mksh and pax). Uploading backports for mksh (jessie and wheezy-sloppy) have been requested by several users, but none of the four(?) DDs asked about sponsoring them even answered at all, and the regular (current) sponsors don t have experience with bpo, so SOL I ve also tweaked a bug in sed(1), in MirBSD. Unfortunately, this means it now comes with the GNUism -i too: don t use it, use ed(1) (much nicer anyway) or perlrun(1) -p/-n Finally, our PDF manpages now use the PA4 paper size instead of DIN ISO A4, meaning they can be printed without cropping or scaling on both A4 and US-american letter paper. And a from the last announcement: we now use Gentium and Inconsolata as body text and monospace fonts, respectively. (And propos, the website ought to be more legible due to text justification and better line spacing now.) I managed to hack this up in GNU groff and Ghostscript, thankfully. (LaTeX too) Currently there are PDF manpages for joe (jupp), mksh, and cpio/pax/tar. And we had Gr nkohl today! Also, new console-setup package in the WTF APT repository since upstream managed to do actual work on it (even fixed some bugs). Read its feed if interested, as its news will not be repeated here usually. (That means, subscribe as there won t be many future reminders in this place.) The netboot.me service appears to be gone. I ll not remove our images, but if someone knows what became of it drop us a message (IRC or mailing list will work just fine). PS: This was originally written on 20160304 but opax refused to be merged in time Happy Birthday, gecko2! In the meantime, the Street Food festival weekend provided wonderful food at BaseCamp, and headache prevented this from being finished on the fifth. Update 06.03.2016: The pax changes were too intrusive, so I decided to only backport the fixes OpenBSD did (both those they mentioned and those silently included), well, the applicable parts of them, anyway, instead. There will be a MirCPIO release completely rebased later after all changes are merged and, more importantly, tested. Another release although not set for immediate future should bring a more sensible (and mksh-like) buildsystem for improved portability (and thus some more changes we had to exclude at first). I ve also cloned the halfwidth part of the FixedMisc [MirOS] font as FixedMiscHW for use with Qt5 applications, xfonts-base in the WTF APT repo. (Debian #809979) tl;dr: mksh R52c (bugfix-only, low-medium); mircpio 20160306 (security backport; high) with future complete rebase (medium) upstream and in Debian. No mksh backports due to lacking a bpo capable sponsor. New console-setup in WTF APT repo, and mksh there as usual. xfonts-base too. netboot.me gone?

5 March 2016

Dirk Eddelbuettel: gunsales 0.1.1

A minor update to our recently released gunsales package is now on the CRAN network. The changes are completely internal and deal with automagic discovery of the X-13ARIMA-SEATS program via our associated x13binary package. More precisely, we let Solaris and the Windows old-release builds fail more gracefully. Others, ie current Windows builds, OS X and Linux, are entirely unaffected in their functionality. For simplicity, all examples now have an \dontrun wrapper, and the vignette makes its code conditional on the platform it is running on. Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

4 March 2016

Mike Gabriel: My FLOSS activities in February 2016

February 2016 has been a very active month regarding me contributing to the FLOSS world. Honouring my Sponsors I am happy to share that this month's FLOSS work has been sponsored by various sponsors. Thanks to all people and companies sponsoring my work on FLOSS projects. This month's MATE uploads to Debian With regards to the Beta 1 Freeze date of Ubuntu 16.04 LTS (18th Feb 2016), Martin Wimpress, Vangelis Mouhtsis and I performed quite some work on Debian MATE. Uploads to Debian unstable: The Debian MATE Packaging Team also took over maintenance of the GTK-2+ legacy package libwnck [13]. The first upload introducing some major changes and package clean-ups caused a slight wave [14] because of a missing dependency in libwnck-dev (that fell victim to some clean-ups in debian/control). Those issues have been addressed immediately and have now been settled. The main reason for working on a legacy package like libwnck was the need for having gir1.2-wnck-1.0 (back) in Debian. The new MATE dock applet requires the libwnck GIR package to be present at runtime. One of the novelties in Ubuntu MATE 16.04 LTS will be the option to adapt the look and feel of the MATE desktop to how a Unity-based desktop looks like. Martin Wimpress is giving intense work to providing a dock applet and topmenu support as one alternative among the various Ubuntu MATE desktop experiences provided. The alternative desktop layouts can be configured with the MATE Tweak tool. Work on RDP related packages Work on FreeRDP 1.1 as currently in Debian I finally managed to give some priority (and thus time) to fixing various issues in the freerdp package in Debian [15]. Many people had provided patches and solutions to open issues and I tried to honour as many of those, as possible. Please note that I had to disable the GStreamer support in FreeRDP for the recent uploads, as the currently used Git snapshot of FreeRDP only supports GStreamer 0.10's API whereas the security team is in the process of having gstreamer0.10-* packages removed from the Debian stretch/unstable archives. Work on FreeRDP 2.0, coming to Debian soon Furthermore, Bernhard Miklautz is currently working on a freerdp2 package, which will bring the latest Git snapshot of FreeRDP upstream into Debian (and also re-introduce GStreamer support, based on GStreamer 1.0). Bernhard invested a lot of time on pushing the current HEAD of FreeRDP upstream [16] towards a FreeRDP 2.x version. Starting with FreeRDP 2.x it will be possible to install different FreeRDP versions on one system without file naming conflicts. For March 2016, I have doing the final freerdp2 reviewing on my todo list (possibly together with H ctor Or n Mart nez who is highly interested in the RDP backend support in Wayland/Weston), so that we can provide first uploads to Debian experimental sometime the coming month. The packaging progress is continuously discussed on the #freerdp channel on Freenode and can also be viewed on Github [17]. Review of revised XRDP package Recently, Dominik George from Teckids e.V. [18] contacted me about reviewing their effort of updating the Debian package xrdp, which currently is in ITA state [19]. Feedback has been provided and I am waiting for a ping from his side so that I can take some (ideally) final looks at the package and sponsor the upload. Work on Debian Edu related packages This month, I spent a couple of hours of work on several Debian Edu related tasks, some of them induced by problems at local school sites we support. Work on Debian LTS My 8h-portion of work for the Debian LTS Project, I performed at the very end of February. With the Debian squeeze LTS EOL date on 29th February, I saw to finalizing my personal open todos regarding Debian squeeze LTS, which basically was getting two CVE issues fixed in the lxc package [26]. The rest of the work hours has been spent on helping out the Security Team of Debian with open CVE issues in Debian wheezy packages: The gosa .debdiff has been approved by a member of the Security Team, the upload will happen today. With my LTS frontdesk hat on (during week 9 / 2016) I also spent some time providing help regarding SVN checkout problems and raised a couple of questions on how to coordinate the work phase between the Debian squeeze LTS EOL and the official launch of the Debian wheezy LTS project phase [27]. Work on nx-libs At the end of February, I finally managed to propose a way of dropping the libNX_Xrender.so bundled library from the nx-libs code base. I filed a PR [28] against nx-libs that proposes its removal and provides a patch for using X.Org's libXrender.so version. As a preview for nx-libs work in March 2016... I have started with removing the complete libNX_X11.so library from nx-libs and using X.Org's X11 client library. This will introduce a code removal of around 160.000 lines of code to nx-libs. More to come on this later... light+love,
Mike [1] http://ubuntu-mate.org/
[2] https://www.freexian.com/
[3] http://www.qindel.com/ [4] (caja)
https://lists.debian.org/debian-devel-changes/2016/02/msg00468.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02080.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02086.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02183.html [5] (mate-menu)
https://lists.debian.org/debian-devel-changes/2016/02/msg00469.html [6] (mate-panel)
https://lists.debian.org/debian-devel-changes/2016/02/msg01900.html [7] (mate-dock-applet)
https://lists.debian.org/debian-devel-changes/2016/02/msg01935.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02481.html
https://lists.debian.org/debian-devel-changes/2016/02/msg03097.html [8] (mate-polkit)
https://lists.debian.org/debian-devel-changes/2016/02/msg01936.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02395.html [9] (eom)
https://lists.debian.org/debian-devel-changes/2016/02/msg02073.html [10] (pluma)
https://lists.debian.org/debian-devel-changes/2016/02/msg02128.html [11] (topmenu-gtk)
https://lists.debian.org/debian-devel-changes/2016/02/msg02399.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02501.html [12] (mate-tweak)
https://lists.debian.org/debian-devel-changes/2016/02/msg03086.html [13] (libwnck)
https://lists.debian.org/debian-devel-changes/2016/02/msg01248.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01404.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01825.html [14] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814585
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814588
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814697 [15] (freerdp)
https://lists.debian.org/debian-devel-changes/2016/02/msg02487.html
https://lists.debian.org/debian-devel-changes/2016/02/msg02630.html [16] https://github.com/FreeRDP/FreeRDP
[17] https://github.com/bmiklautz/debian-freerdp2 [18] https://www.teckids.org/ [19] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=719624 [20] (gosa)
https://lists.debian.org/debian-devel-changes/2016/02/msg01554.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01954.html [21] https://sunweavers.net/blog/node/34 [22] (ldap2zone)
https://lists.debian.org/debian-devel-changes/2016/02/msg01966.html
https://lists.debian.org/debian-devel-changes/2016/02/msg01967.html [23] (shutdown-at-night)
https://lists.debian.org/debian-devel-changes/2016/02/msg03605.html [24] (italc)
https://lists.debian.org/debian-devel-changes/2016/02/msg01944.html [25] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=815948 [26] (lxc, Debian squeeze LTS)
https://lists.debian.org/debian-lts-changes/2016/02/msg00037.html [27] https://lists.debian.org/debian-lts/2016/02/msg00155.html
(The thread continues in March 2016) [28] https://github.com/ArcticaProject/nx-libs/pull/93

3 March 2016

Simon Richter: Why sysvinit?

As most distributions have switched to systemd as default "init", some people have asked why we actually keep sysvinit around, as it's old and crusty, and systemd can do so much more. My answer is: systemd and sysvinit solve entirely different problems, and neither can ever replace the other fully. The big argument in favour of systemd is integration: when you keep everything in one process, it is easy to make sure that components talk to each other. That is nice, but kind of misses the point: integration could be done via IPC as well, and systemd uses a lot of it to separate privileges anyway. The tighter integration comes from something else: the configuration format. Where sysvinit uses a lot of shell scripts ("imperative" format), systemd uses explicit keywords ("descriptive" format), and this is the important design decision here: descriptive formats cannot do anything unanticipated, neither good nor bad. This is important for UI programmers: if there is a button that says "if I close the lid switch, suspend the laptop", then this needs to be connected to functionality that behaves this way, and this connection needs to work in both directions, so the user will find the active setting there when opening the dialog. So, we need to limit ourselves to a closed set of choices, so the UI people can prepare translations. This is the design tradeoff systemd makes: better integration in exchange for less flexibility. Of course, it is simple to allow "custom" actions everywhere, but that starts to break the "well-integrated" behaviour. There is an old blog post from Joey Hess about how to implement a simple alarm clock, using systemd, and this shows both how powerful and how limited the system is. On one hand, the single line WakeSystem=true takes care of the entire dependency chain attached to making sure the system is actually on -- when the system is going to suspend mode, some program needs to determine that there are scheduled jobs that should wake up the system, determined which of these is the next one, program the wakeup time into the hardware and only then allow the suspend to continue. On the other hand, the descriptive framework already breaks down in this simple example, because the default behaviour of the system is to go to sleep if the lid switch is closed, and the easiest way to fix this is to tell systemd to ignore the lid switch while the program is running. The IPC permissions disallow scheduled user jobs from setting this, so a "custom" action is taken that sets up the "ignore the lid switch" policy, changes to an unprivileged user, and runs the actual job. So, it is possible to get back flexibility, but this is traded for integration: while the job is running, any functionality that the "suspend laptop when the lid is closed" button had, is broken. For something simple as a switch on a personal laptop that breaks if the user configures something that is non-standard, that is acceptable. However, the keywords in the service unit files, the timer files, etc. are also part of a descriptive framework, and the services using them expect that the things they promise are kept by the framework, so there is a limit to how many exceptions you can grant. SystemV init really comes from the other direction here. Nothing is promised to init scripts but a vague notion that they will be run with sufficient privileges. There is no dependency tracking that ensures that certain services will always start before others, just priorities. You can add a dependency system like insserv if you like and make sure that all the requirements for it to work (i.e. services declaring their dependencies) are given, and there is a safe fallback of not parallelizing so we can always play it safe. Because there is so little promised to scripts, writing them becomes a bit tedious -- all of them have a large case statement where they parse the command line, all of them need to reimplement dropping privileges and finding the PID of a running process. There are lots of helpers for the more common things, like the LSB standard functions for colorful and matching console output, and Debian's start-stop-daemon for privilege and PID file handling, but in the end it remains each script's responsibility. Systemd will never be able to provide the same flexibility to admins while at the same time keeping the promises made in the descriptive language, and sysvinit will never reach the same amount of easy integration. I think it is fairly safe to predict both of these things, and I'd even go a step further: if any one of them tried, I'd ask the project managers what they were thinking. The only way to keep systems as complex as these running is to limit the scope of the project. With systemd, the complexity is kept in the internals, and it is important for manageability that all possible interactions are well-defined. That is a huge issue when writing adventure games, where you need to define interaction rules for each player action in combination with each object, and each pair of objects that can be combined with the "Use" action. Where adventure games would default to "This doesn't seem to work." or "I cannot use this.", this would not be a helpful error message when we're trying to boot, so we really need to cut down on actions and object types here. Sysvinit, on the other hand, is both more extreme in limiting the scope to very few actions (restart, wait, once) and only one object type (process that can be started and may occasionally terminate) in the main program, and a lot more open in what scripts outside the main program are allowed to do. The prime example for this is that the majority of configuration takes place outside the inittab even -- a script is responsible for the interface that services are registered by dropping init scripts into specific directories, and scripts can also create additional interfaces, but none of this is intrinsic to the init system itself. Which system is the right choice for your system is just that: your choice. You would choose a starting point, and customize what is necessary for your computer to do what you want it to do. Here's the thing: most users will be entirely happy with fully uncustomized systemd. It will suspend your laptop if you close the lid, and even give your download manager veto power. I fully support Debian's decision to use systemd as the default init for new installs, because it makes sense to use the default that is good for the vast majority of users. However, my opening statement still stands: systemd can never support all use cases, because it is constrained by its design. Trying to change that would turn it into an unmaintainable mess, and the authors are right in rejecting this. There are use cases that define the project, and use cases that are out of scope. This is what we need sysvinit for: the bits that the systemd project managers rightfully reject, but that are someone's use case. Telling these users to get lost would be extremely bad style for the "universal operating system." For example, if someone needs a service that asks a database server for a list of virtual machines to start, runs each in its private network namespace that is named after replacing part of the virtual machine name with the last two octets of the host machine's IP address, then binds a virtual Ethernet device into the network namespace and sets up a NAT rule that allows devices to talk to the public Internet and a firewall rule to stop VMs from talking to each other. Such a beast would live outside of systemd's world view. You can easily start it, but systemd would not know how to monitor it (as long as there is some process still running, is that a good sign), not know how to shut down one instance, not know how to map processes to instances and so on. SystemV init has the advantage here that none of these things are defined by the system, and as long as my init script has some code to handle "status", I can easily check up on my processes with the same interface I'd use for everything else. This requires me to have more knowledge of how the init system works, that is true, however I'd also venture that the learning curve for sysvinit is shallower. If you know shell scripting, you can understand what init scripts do, without memorizing special rules that govern how certain keywords interact, and keeping track of changes to these rules as the feature set is expanded. That said, I'm looking at becoming the new maintainer of the sysvinit package so we can continue to ship it and give our users a choice, but I'd definitely need help, as I only have a few systems left that won't run properly with systemd because of its nasty habit to keep lots of code in memory where sysvinit would unload it after use (by letting the shell process end), and these don't make good test systems. The overall plan is to set up a CI system that Ideally, I'd like to reach a state where we can actually ensure that the default configuration is somewhat similar regardless of which init system we use, and that it is possible to install with either (this is a nasty regression in jessie: bootstrapping a foreign-architecture system broke). This will take time, and step one is probably looking at bugs first -- I'm grateful for any help I can get here, primarily I'd like to hear about regressions via bug reports. tl;dr: as a distribution, we need both systemd and sysvinit. If you suggest that systemd can replace sysvinit, I will probably understand that you think that the systemd maintainers don't know project management. I'm reluctantly looking at becoming the official maintainer because I think it is important to keep around.

Next.

Previous.