Recently I was writing log analysis tools in javascript. Javascript part is challenging.
Here is my monthly update covering what I have been doing in the free software world in September 2017 (previous month):
reproducible-check is our script to determine which packages actually installed on your system are reproducible or not.
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.
disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues.
My website and blog are managed by the static site generator
Hugo. I ve found this to be a stable and flexible system,
but at the last upgrade a breaking change has occurred that broken the
syndication of my blog on various planets.
At first I thought perhaps with my increased posting rate the planets were
truncating my posts but this was not the case. The problem was in Hugo pull
request #3129
where for some reason they have changed the RSS feed to contain only a lead
instead of the full article.
I ve seen other content management systems offer a similar option but at least
they point out that it s truncated and offer a read more link. Here it just
looks like I m publishing truncated unfinished really short posts.
If you take a look at the post above, you ll see that the change is in an
embedded template and it took a little reading the docs to work out how to
revert the change. The steps are actually not that difficult, but it s still
annoying that the change occurred.
In a Hugo site, you will have a layouts directory that will contain your
overrides from your theme. Create a new file in the path
layouts/_default/rss.xml (you may need to create the _default directory)
with the following content:
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title> if eq .Title .Site.Title .Site.Title else with .Title . on end .Site.Title end </title>
<link> .Permalink </link>
<description>Recent content if ne .Title .Site.Title with .Title in . end end on .Site.Title </description>
<generator>Hugo -- gohugo.io</generator> with .Site.LanguageCode
<language> . </language> end with .Site.Author.email
<managingEditor> . with $.Site.Author.name ( . ) end </managingEditor> end with .Site.Author.email
<webMaster> . with $.Site.Author.name ( . ) end </webMaster> end with .Site.Copyright
<copyright> . </copyright> end if not .Date.IsZero
<lastBuildDate> .Date.Format "Mon, 02 Jan 2006 15:04:05 -0700" safeHTML </lastBuildDate> end
with .OutputFormats.Get "RSS"
printf "<atom:link href=%q rel=\"self\" type=%q />" .Permalink .MediaType safeHTML
end
range .Data.Pages
<item>
<title> .Title </title>
<link> .Permalink </link>
<pubDate> .Date.Format "Mon, 02 Jan 2006 15:04:05 -0700" safeHTML </pubDate>
with .Site.Author.email <author> . with $.Site.Author.name ( . ) end </author> end
<guid> .Permalink </guid>
<description> .Content html </description>
</item>
end
</channel>
</rss>
The thirteenth release in the 0.12.* series of Rcpp landed on CRAN this morning, following a little delay because Uwe Ligges was traveling and whatnot. We had announced its availability to the mailing list late last week. As usual, a rather substantial amount of testing effort went into this release so you should not expect any surprise.
This release follows the 0.12.0 release from July 2016, the 0.12.1 release in September 2016, the 0.12.2 release in November 2016, the 0.12.3 release in January 2017, the 0.12.4 release in March 2016, the 0.12.5 release in May 2016, the 0.12.6 release in July 2016, the 0.12.7 release in September 2016, the 0.12.8 release in November 2016, the 0.12.9 release in January 2017, the 0.12.10.release in March 2017, the 0.12.11.release in May 2017, and the 0.12.12 release in July 2017 making it the seventeeth release at the steady and predictable bi-montly release frequency.
Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 1069 packages (and hence 73 more since the last release) on CRAN depend on Rcpp for making analytical code go faster and further, along with another 91 in BioConductor.
This releases contains a large-ish update to the documentation as all vignettes (apart from the unit test one, which is a one-off) now use Markdown and the (still pretty new) pinp package by James and myself. There is also a new vignette corresponding to the PeerJ preprint James and I produced as an updated and current Introduction to Rcpp replacing the older JSS piece (which is still included as a vignette too).
A few other things got fixed: Dan is working on const iterators you would expect with modern C++, Lei Yu spotted error in Modules, and more. See below for details.
Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.Changes in Rcpp version 0.12.13 (2017-09-24)
- Changes in Rcpp API:
- Changes in Rcpp Modules:
- Misplacement of one parenthesis in macro
LOAD_RCPP_MODULEwas corrected (Lei Yu in #737)- Changes in Rcpp Documentation:
- Rewrote the macOS sections to depend on official documentation due to large changes in the macOS toolchain. (James Balamuta in #742 addressing issue #682).
- Added a new vignette Rcpp-introduction based on new PeerJ preprint, renamed existing introduction to Rcpp-jss-2011 .
- Transitioned all vignettes to the 'pinp' RMarkdown template (James Balamuta and Dirk Eddelbuettel in #755 addressing issue #604).
- Added an entry on running 'compileAttributes()' twice to the Rcpp-FAQ (##745).
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
.path units
This kind of unit can be used to monitor a file or directory for changes using
inotify, and activate other units when
an event happens.
For example, this activates a unit that manages a spool directory, which
activates another unit whenever a .pdf file is added to /tmp/spool/:
[Unit] Description=Monitor /tmp/spool/ for new .pdf files [Path] Unit=beeponce.service PathExistsGlob=/tmp/spool/*.pdf MakeDirectory=true
/tmp/ready is changed, for
example by someone running touch /tmp/ready:
[Unit] Description=Monitor /tmp/ready [Path] Unit=beeponce.service PathChanged=/tmp/ready
beeponce.service:
[Unit] Description=Beeps once [Service] Type=oneshot ExecStart=/usr/bin/aplay /tmp/beep.wav
man systemd.path
ast year, the AppStream specification gained proper support for adding metadata for fonts, after Richard Hughes did some work on it years ago. We weren t happy with how fonts were handled at that time, so we searched for better solutions, which is why this took a bit longer to be done. Last year, I was implementing the final support for fonts in both appstream-generator (the metadata extractor used by Debian and a few others) as well as the AppStream specification. This blogpost was sitting on my todo list as a draft for a long time now, and I only just now managed to finish it, so sorry for announcing this so late. Fonts are already available via AppStream for a year, and this post just sums up the status quo and some neat tricks if you want to write metainfo files for fonts. If you are following AppStream (or the Debian fonts list), you know everything already 
.
Both Richard and I first tried to extract all the metadata to display fonts in a proper way to the users from the font files directly. This turned out to be very difficult, since font metadata is often wrong or incomplete, and certain desirable bits of metadata (like a longer description) are missing entirely. After messing around with different ways to solve this for days (afterall, by extracting the data from font files directly we would have hundreds of fonts directly available in software centers), I also came to the same conclusion as Richard: The best and easiest solution here is to mandate the availability of metainfo files per font.
Which brings me to the second issue: What is a font? For any person knowing about fonts, they will understand one font as one font face, e.g. Lato Regular Italic or Lato Bold . A user however will see the font family as a font, e.g. just Lato instead of all the font faces separated out. Since AppStream data is used primarily by software centers, we want something that is easy for users to understand. Hence, an AppStream font components really describes a font family or collection of fonts, instead of individual font faces. We do also want AppStream data to be useful for system components looking for a specific font, which is why font components will advertise the individual font face names they contain via a <provides/>-tag. Naming fonts and making them identifiable is a whole other issue, I used a document from Adobe on font naming issues as a rough guideline while working on this. How to write a good metainfo file for a font is best shown with an example. Lato is a well-looking font family that we want displayed in a software center. So, we write a metainfo file for it an place it in
/usr/share/metainfo/com.latofonts.Lato.metainfo.xmlfor the AppStream metadata generator to pick up:
<?xml version="1.0" encoding="UTF-8"?>
<component type="font">
<id>com.latofonts.Lato</id>
<metadata_license>FSFAP</metadata_license>
<project_license>OFL-1.1</project_license>
<name>Lato</name>
<summary>A sanserif type face fam ily</summary>
<description>
<p>
Lato is a sanserif type face fam ily designed in the Sum mer 2010 by Warsaw-based designer
ukasz Dziedzic ( Lato means Sum mer in Pol ish). In Decem ber 2010 the Lato fam ily
was pub lished under the open-source Open Font License by his foundry tyPoland, with
sup port from Google.
</p>
</description>
<url type="homepage">http://www.latofonts.com/</url>
<provides>
<font>Lato Regular</font>
<font>Lato Black Italic</font>
<font>Lato Black</font>
<font>Lato Bold Italic</font>
<font>Lato Bold</font>
<font>Lato Hairline Italic</font>
...
</provides>
</component>When the file is processed, we know that we need to look for fonts in the package it is contained in. So, the appstream-generator will load all the fonts in the package and render example texts for them as an image, so we can show users a preview of the font. It will also use heuristics to render an icon for the respective font component using its regular typeface. Of course that is not ideal what if there are multiple font faces in a package? What if the heuristics fail to detect the right font face to display?
This behavior can be influenced by adding <font/>tags to a
<provides/>tag in the metainfo file. The font-provides tags should contain the fullnames of the font faces you want to associate with this font component. If the font file does not define a fullname, the family and style are used instead. That way, someone writing the metainfo file can control which fonts belong to the described component. The metadata generator will also pick the first mentioned font name in the
<provides/>list as the one to render the example icon for. It will also sort the example text images in the same order as the fonts are listed in the provides-tag. The example lines of text are written in a language matching the font using Pango. But what about symbolic fonts? Or fonts where any heuristic fails? At the moment, we see ugly tofu characters or boxes instead of an actual, useful representation of the font. This brings me to an inofficial extension to font metainfo files, that, as far as I know, only appstream-generator supports at the moment. I am not happy enough with this solution to add it to the real specification, but it serves as a good method to fix up the edge cases where we can not render good example images for fonts. AppStream-Generator supports the FontIconText and FontSampleText custom AppStream properties to allow metainfo file authors to override the default texts and autodetected values. FontIconText will override the characters used to render the icon, while FontSampleText can be a line of text used to render the example images. This is especially useful for symbolic fonts, where the heuristics usually fail and we do not know which glyphs would be representative for a font. For example, a font with mathematical symbols might want to add the following to its metainfo file:
<custom> <value key="FontIconText"> </value> <value key="FontSampleText"> ... </value> </custom>Any unicode glyphs are allowed, but asgen will but some length restrictions on the texts. So, In summary:
I fixed a bug in Launchpad recently that led me deeper than I expected.
Launchpad uses Buildout as its build system for
Python packages, and it s served us well for many years. However, we re
using 1.7.1, which doesn t support ensuring that packages required using
setuptools setup_requires
keyword
only ever come from the local index URL when one is specified; that s an
essential constraint we need to be able to impose so that our build system
isn t immediately sensitive to downtime or changes in PyPI. There are
various issues/PRs about this in Buildout (e.g.
#238), but even if those
are fixed it ll almost certainly only be in Buildout v2, and upgrading to
that is its own kettle of fish for other reasons. All this is a serious
problem for us because newer versions of many of our vital dependencies
(Twisted and
testtools, to name but two) use
setup_requires to pull in pbr, and so
we ve been stuck on old versions for some time; this is part of why
Launchpad doesn t yet support newer SSH key types, for instance. This
situation obviously isn t sustainable.
To deal with this, I ve been working for some time on switching to
virtualenv and
pip. This is harder than you might think:
Launchpad is a long-lived and complicated project, and it had quite a number
of explicit and implicit dependencies on Buildout s configuration and
behaviour. Upgrading our infrastructure from Ubuntu 12.04 to 16.04 has
helped a lot (12.04 s baseline virtualenv and pip have some deficiencies
that would have required a more complicated bootstrapping procedure). I ve
dealt with most of these: for example, I had to reorganise a lot of our
helper scripts
(1,
2,
3),
but there are still a few more things to go.
One remaining problem was that our Buildout configuration relied on building
several different environments with different Python paths for various
things. While this would technically be possible by way of building
multiple virtualenvs, this would inflate our build time even further (we re
already going to have to cope with some slowdown as a result of using
virtualenv, because the build system now has to do a lot more than
constructing a glorified link farm to a bunch of cached eggs), and it seems
like unnecessary complexity. The obvious thing to do seemed to be to
collapse these into a single environment, since there was no obvious reason
why it should actually matter if
txpkgupload and
txlongpoll were carefully kept
off the path when running most of Launchpad: so I did
that.
Then our build system got very
sad.
Hmm, I thought. To keep our test times somewhat manageable, we run them in
parallel across 20 containers, and we randomise the order in which they run
to try to shake out test isolation bugs. It s not completely unknown for
there to be some oddities resulting from that. So I ran it again. Nope,
but slightly differently sad this
time.
Furthermore, I couldn t reproduce these failures locally no matter how hard
I tried. Oh dear. This was obviously not going to be a good day.
In fact I spent a while on various different guesswork-based approaches. I
found bug 571334 in
Ampoule, an AMP-based process pool implementation that we use for some job
runners, and proposed a
fix
for that, but cherry-picking that fix into Launchpad didn t help matters. I
tried backing out subsets of my changes and determined that if both
txlongpoll and txpkgupload were absent from the Python module path in
the context of the tests in question then everything was fine. I tried
running strace locally and staring at the output for some time in the hope
of enlightenment: that reminded me that the two packages in question install
modules under twisted.plugins, which did at least establish a reason they
might affect the environment that was more plausible than magic, but nothing
much more specific than that.
On Friday I was fiddling about with this again and trying to insert some
more debugging when I noticed some interesting behaviour around plugin
caching.
If I caused the txpkgupload plugin to raise an exception when loaded, the
Twisted plugin system would remove its dropin.cache (because it was stale)
and not create a new one (because there was now no content to put in it).
After that, running the relevant tests would fail as I d seen in our
buildbot. Aha! This meant that I could also reproduce it by doing an even
cleaner build than I d previously tried to do, by removing the cached
txpkgupload and txlongpoll eggs and allowing the build system to
recreate them. When they were recreated, they didn t contain
dropin.cache, instead allowing that to be created on first use.
Based on this clue I was able to get to the answer relatively quickly.
Ampoule has a specialised bootstrapping sequence for its worker processes
that starts by doing this:
from twisted.application import reactors
reactors.installReactor(reactor)
twisted.application.reactors.installReactor calls
twisted.plugin.getPlugins, so the very start of this bootstrapping
sequence is going to involve loading all plugins found on the module path (I
assume it s possible to write a plugin that adds an alternative reactor
implementation). If dropin.cache is up to date, then it will just get the
information it needs from that; but if it isn t, it will go ahead and import
the plugin. If the plugin happens (as Twisted code often does) to run from
twisted.internet import reactor at some point while being imported, then
that will install the platform s default reactor, and then
twisted.application.reactors.installReactor will raise
ReactorAlreadyInstalledError. Since Ampoule turns this into an info-level
log message for some reason, and the tests in question only passed through
error-level messages or higher, this meant that all we could see was that a
worker process had exited non-zero but not why.
The Twisted documentation
recommends
generating the plugin cache at build time for other reasons, but we weren t
doing that. Fixing
that
makes everything work again.
There are still a few more things needed to get us onto pip, but we re now
pretty close. After that we can finally start bringing our dependencies up
to date.
I ve received an email today from Barclaycard with the following:
From time to time, to make sure it s you who s using your Barclaycard online, we ll send you a text with a verification code for you to use on the Verified by Visa screen that ll pop up on your payment page.The proprietary nature of mobile phones with the hardware specifications and the software being closed off from inspection or audit and considered to be trade secrets make my phone and my tablet the least trusted devices I own and use. Due to this lack of trust, I ve often held back from using my phone or tablet for certain tasks where I can still get away with not doing so. I have experimented with having read-only access to my calendars and contacts to ensure that if my phone is compromised they can t just be wiped out, though in the end I had to give in as my calendar was becoming too difficult to manage using a paper system as part of entry for new events. I wanted to try to reduce the attractiveness of compromising my phone. Anyone that really wants to have a go at my phone could probably get in. It s an older Samsung Android phone on a UK network and software updates rarely come through in a timely manner. Anything that I give my phone access to is at risk and that risk needs to be balanced by some real world benefits. These are just the problems with the phone itself. When you re using SMS authentication, even with the most secure phone ever, you re still going to be using the phone network. SMS authentication is about equivalent, in terms of the security it really offers, to your mobile phone number being your password when it comes to an even mildly motivated attacker. You probably don t treat your mobile phone number as a password, nor does the provider or anyone you ve given it to, so you can assume that it s compromised. Why are mobile phones so popular for two factor (on in increasing numbers of cases, single factor) authentication? Not because they improve security but because they re convenient and everyone has one. This seems like a bad plan.
Lintian is a static analysis tool for Debian packages, reporting on various errors, omissions and general quality-assurance issues to maintainers.
I've previously written about my exploits with Lintian as well as authoring a short tutorial on how to write your own Lintian check.
Anyway, I recently uploaded version 2.5.53 about two months since previous release. The biggest changes you may notice are supporting the latest version of the Debian Policy as well the addition of checks to encourage the migration to Python 3.
Thanks to all who contributed patches, code review and bug reports to this release. The full changelog is as follows:
lintian (2.5.53) unstable; urgency=medium
The "we are all Perl developers now" release.
* Summary of tag changes:
+ Added:
- alternatively-build-depends-on-python-sphinx-and-python3-sphinx
- build-depends-on-python-sphinx-only
- dependency-on-python-version-marked-for-end-of-life
- maintainer-script-interpreter
- missing-call-to-dpkg-maintscript-helper
- node-package-install-in-nodejs-rootdir
- override-file-in-wrong-package
- package-installs-java-bytecode
- python-foo-but-no-python3-foo
- script-needs-depends-on-sensible-utils
- script-uses-deprecated-nodejs-location
- transitional-package-should-be-oldlibs-optional
- unnecessary-testsuite-autopkgtest-header
- vcs-browser-links-to-empty-view
+ Removed:
- debug-package-should-be-priority-extra
- missing-classpath
- transitional-package-should-be-oldlibs-extra
* checks/apache2.pm:
+ [CL] Fix an apache2-unparsable-dependency false positive by allowing
periods (".") in dependency names. (Closes: #873701)
* checks/binaries.pm:
+ [CL] Apply patches from Guillem Jover & Boud Roukema to improve the
description of the binary-file-built-without-LFS-support tag.
(Closes: #874078)
* checks/changes. pm,desc :
+ [CL] Ignore DFSG-repacked packages when checking for upstream
source tarball signatures as they will never match by definition.
(Closes: #871957)
+ [CL] Downgrade severity of orig-tarball-missing-upstream-signature
from "E:" to "W:" as many common tools do not make including the
signatures easy enough right now. (Closes: #870722, #870069)
+ [CL] Expand the explanation of the
orig-tarball-missing-upstream-signature tag to include the location
of where dpkg-source will look. Thanks to Theodore Ts'o for the
suggestion.
* checks/copyright-file.pm:
+ [CL] Address a number of issues in copyright-year-in-future:
- Prevent false positives in port numbers, email addresses, ISO
standard numbers and matching specific and general street
addresses. (Closes: #869788)
- Match all violating years in a line, not just the first (eg.
"2000-2107").
- Ignore meta copyright statements such as "Original Author". Thanks
to Thorsten Alteholz for the bug report. (Closes: #873323)
- Expand testsuite.
* checks/cruft. pm,desc :
+ [CL] Downgrade severity of file-contains-fixme-placeholder
tag from "important" (ie. "E:") to "wishlist" (ie. "I:").
Thanks to Gregor Herrmann for the suggestion.
+ [CL] Apply patch from Alex Muntada (alexm) to use "substr" instead
of "substring" in mentions-deprecated-usr-lib-perl5-directory's
description. (Closes: #871767)
+ [CL] Don't check copyright_hints file for FIXME placeholders.
(Closes: #872843)
+ [CL] Don't match quoted "FIXME" variants as they are almost always
deliberate. Thanks to Adrian Bunk for the report. (Closes: #870199)
+ [CL] Avoid false positives in missing source checks for "CSS Browser
Selector". (Closes: #874381)
* checks/debhelper.pm:
+ [CL] Prevent a false positive of
missing-build-dependency-for-dh_-command that can be exposed by
following the advice for the recently added
useless-autoreconf-build-depends tag. (Closes: #869541)
* checks/debian-readme. pm,desc :
+ [CL] Ensure readme-debian-contains-debmake-template also checks
for templates "Automatically generated by debmake".
* checks/description. desc,pm :
+ [CL] Clarify explanation of description-starts-with-leading-spaces
tag. Thanks to Taylor Kline for the report
and patch. (Closes: #849622)
+ [NT] Skip capitalization-error-in-description-synopsis for
auto-generated packages (such as dbgsym packages).
* checks/fields. desc,pm :
+ [CL] Ensure that python3-foo packages have "Section: python", not
just python2-foo. (Closes: #870272)
+ [RG] Do no longer require debug packages to be priority extra.
+ [BR] Use Lintian::Data for name/section mapping
+ [CL] Check for packages including "?rev=0&sc=0" in Vcs-Browser.
(Closes: #681713)
+ [NT] Transitional packages should now be "oldlibs/optional" rather
than "oldlibs/extra". The related tag has been renamed accordingly.
* checks/filename-length.pm:
+ [NT] Skip the check on auto-generated binary packages (such as
dbgsym packages).
* checks/files. pm,desc :
+ [BR] Avoid privacy-breach-generic false positives for legal.xml.
+ [BR] Detect install of node package under /usr/lib/nodejs/[^/]*$
+ [CL] Check for packages shipping compiled Java class files. Thanks
Carn Draug . (Closes: #873211)
+ [BR] Privacy breach is no longer experimental.
* checks/init.d.desc:
+ [RG] Do not recommend a versioned dependency on lsb-base in
init.d-script-needs-depends-on-lsb-base. (Closes: #847144)
* checks/java.pm:
+ [CL] Additionally consider .cljc files as code to avoid false-
positive codeless-jar warnings. (Closes: #870649)
+ [CL] Drop problematic missing-classpath check. (Closes: #857123)
* checks/menu-format.desc:
+ [CL] Prevent false positives in desktop-entry-lacks-keywords-entry
for "Link" and "Directory" .desktop files. (Closes: #873702)
* checks/python. pm,desc :
+ [CL] Split out Python checks from "scripts" check to a new, source
check of type "source".
+ [CL] Check for python-foo without corresponding python3-foo packages
to assist in Python 2.x deprecation. (Closes: #870681)
+ [CL] Check for packages that Build-Depend on python-sphinx only.
(Closes: #870730)
+ [CL] Check for packages that alternatively Build-Depend on the
Python 2 and Python 3 versions of Sphinx. (Closes: #870758)
+ [CL] Check for binary packages that depend on Python 2.x.
(Closes: #870822)
* checks/scripts.pm:
+ [CL] Correct false positives in
unconditional-use-of-dpkg-statoverride by detecting "if !" as a
valid shell prefix. (Closes: #869587)
+ [CL] Check for missing calls to dpkg-maintscript-helper(1) in
maintainer scripts. (Closes: #872042)
+ [CL] Check for packages using sensible-utils without declaring a
dependency after its split from debianutils. (Closes: #872611)
+ [CL] Warn about scripts using "nodejs" as an interpreter now that
nodejs provides /usr/bin/node. (Closes: #873096)
+ [BR] Add a statistic tag giving interpreter.
* checks/testsuite. desc,pm :
+ [CL] Remove recommendations to add a "Testsuite: autopkgtest" field
to debian/control as it is added when needed by dpkg-source(1)
since dpkg 1.17.1. (Closes: #865531)
+ [CL] Warn if we see an unnecessary "Testsuite: autopkgtest" header
in debian/control.
+ [NT] Recognise "autopkgtest-pkg-go" as a valid test suite.
+ [CL] Recognise "autopkgtest-pkg-elpa" as a valid test suite.
(Closes: #873458)
+ [CL] Recognise "autopkgtest-pkg-octave" as a valid test suite.
(Closes: #875985)
+ [CL] Update the description of unknown-testsuite to reflect that
"autopkgtest" is not the only valid value; the referenced URL
is out-of-date (filed as #876008). (Closes: #876003)
* data/binaries/embedded-libs:
+ [RG] Detect embedded copies of heimdal, libgxps, libquicktime,
libsass, libytnef, and taglib.
+ [RG] Use an additional string to detect embedded copies of
openjpeg2. (Closes: #762956)
* data/fields/name_section_mappings:
+ [BR] node- package section is javascript.
+ [CL] Apply patch from Guillem Jover to add more section mappings.
(Closes: #874121)
* data/fields/obsolete-packages:
+ [MR] Add dh-systemd. (Closes: #872076)
* data/fields/perl-provides:
+ [CL] Refresh perl provides.
* data/fields/virtual-packages:
+ [CL] Update data file from archive. This fixes a false positive for
"bacula-director". (Closes: #835120)
* data/files/obsolete-paths:
+ [CL] Add note to /etc/bash_completion.d entry regarding stricter
filename requirements. (Closes: #814599)
* data/files/privacy-breaker-websites:
+ [BR] Detect custom donation logos like apache.
+ [BR] Detect generic counter website.
* data/standards-version/release-dates:
+ [CL] Add 4.0.1 and 4.1.0 as known standards versions.
(Closes: #875509)
* debian/control:
+ [CL] Mention Debian Policy v4.1.0 in the description.
+ [CL] Add myself to Uploaders.
+ [CL] Drop unnecessary "Testsuite: autopkgtest"; this is implied from
debian/tests/control existing.
* commands/info.pm:
+ [CL] Add a --list-tags option to print all tags Lintian knows about.
Thanks to Rajendra Gokhale for the suggestion. (Closes: #779675)
* commands/lintian.pm:
+ [CL] Apply patch from Maia Everett to avoid British spelling when
using en_US locale. (Closes: #868897)
* lib/Lintian/Check.pm:
+ [CL] Stop emitting maintainer,uploader -address-causes-mail-loops
for @packages.debian.org addresses. (Closes: #871575)
* lib/Lintian/Collect/Binary.pm:
+ [NT] Introduce an "auto-generated" argument for "is_pkg_class".
* lib/Lintian/Data.pm:
+ [CL] Modify Lintian::Data's "all" to always return keys in insertion
order, dropping dependency on libtie-ixhash-perl.
* helpers/coll/objdump-info-helper:
+ [CL] Apply patch from Steve Langasek to accommodate binutils 2.29
outputting symbols in a different format on ppc64el.
(Closes: #869750)
* t/tests/fields-perl-provides/tags:
+ [CL] Update expected output to match new Perl provides.
* t/tests/files-privacybreach/*:
+ [CL] Add explicit test for packages including external fonts via
the Google Font API. Thanks to Ian Jackson for the report.
(Closes: #873434)
+ [CL] Add explicit test for packages including external fonts via
the Typekit API via <script/> HTML tags.
* t/tests/*/desc:
+ [CL] Add missing entries in "Test-For" fields to make
development/testing workflow less error-prone.
* private/generate-tag-summary:
+ [CL] git-describe(1) will usually emit 7 hexadecimal digits as the
abbreviated object name, However, as this can be user-dependent,
pass --abbrev=0 to ensure it does not vary between systems. This
also means we do not need to strip it ourselves.
* private/refresh-*:
+ [CL] Use deb.debian.org as the default mirror.
+ [CL] Update locations of Contents-<arch> files; they are now
namespaced by distribution (eg. "main").
-- Chris Lamb <lamby@debian.org> Wed, 20 Sep 2017 09:25:06 +0100
.service units
Describe how to start and stop system services, like daemons.
Services are described in a [Service] section. The Type= configuration
describes how the service wants to be brought up:
Type=simple: a program is started and runs forever, providing the servive.
systemd takes care of daemonizing it properly in the background, creating a
pidfile, stopping it and so on. The service is considered active as soon as
it has started.Type=forking: a traditional daemons that forks itself, creates a pidfile
and so on. The server is considered active as soon as the parent process
ends.Type=oneshot: a program is run once, and the service is considered started
after the program ends. This can be used, for example, to implement a service
to do one-off configuration, like checking a file system.Type=dbus: like simple but for D-Bus services: the service is considered
active as soon as it appears on the D-Bus bus.Type=notify: like simple, but the service tells sytemd when it has
finished initialization and is ready. Notification can happen via the
sd_notify
C function, or the
systemd-notify
command.Type=idle: like simple, but it is run after all other services has been
started on a transaction. You can use this, for example, to start a shell on
a terminal after the boot, so that the prompt doesn't get flooded with boot
messages, or to play a happy trumped sound after the system has finished
booting.man systemd.service,
man systemd.exec,
man systemd.resource-control, and
man systemd.kill.
See systemctl --all -t service for examples.
.unit files
For reference, the global index with all .unit file directives is at
man systemd.directives.
All unit files have a [Unit] section with documentation and dependencies. See
man systemd.unit
for documentation.
It is worth having a look at existing units to see what they are like. Use
systemctl --all -t unittype for a list, and systemctl cat unitname to see
its content wherever it is installed.
For example: systemctl cat graphical.target. Note that systemctl cat adds a
line of comment at the top so one can see where the unit file is installed.
Most unit files also have an [Install] section (also documented in
man systemd.unit)
that controls what happens when enabling or disabling the unit.
See also:
.target units
.target units only contain [Unit] and [Install] sections, and can be used
to give a name to a given set of dependencies.
For example, one could create a remote-maintenance.target unit, that when
brought up activates, via dependencies, a set of services, mounts, network
sockets, and so on.
See man systemd.target
See systemctl --all -t target for examples.
special units
man systemd.special
has a list of units names that have a standard use associated to them.
For example, ctrl-alt-del.target is a unit that is started whenever
Control+Alt+Del is pressed on the console. By default it is symlinked to
reboot.target, and you can provide your own version in /etc/systemd/system/
to perform another action when Control+Alt+Del is pressed.
User units
systemd can also be used to manage services on a user session, starting them at
login and stopping them at logout.
Add --user to the normal systemd commands to have them work with the current
user's session instead of the general system.
See systemd/User in the Arch Wiki
for a good description of what it can do.
systemctl status [unitname [unitname..]] show status of one or more units,
or of the whole system. Glob patterns also work: systemctl status "systemd-fsck@*"systemctl list-units or just systemctl show a table with all units, their
status and their descriptionsystemctl list-sockets lists listening sockets managed by systemd and what
they activatesystemctl list-timers lists timer-activated units, with information about
when they last ran and when they will run againsystemctl is-active [pattern] checks if one or more units are in active statesystemctl is-enabled [pattern] checks if one or more units are enabledsystemctl is-failed [pattern] checks if one or more units are in failed statesystemctl list-dependencies [unitname] lists the dependencies of a unit, or
a system-wide dependency treesystemctl is-system-running check if the system is running correctly, or if
some unit is in a failed statesystemd-cgtop like top
but processes are aggregated by unitsystemd-analyze produces reports on boot time, per-unit boot time charts,
dependency graphs, and moreservice command,
systemctl provides commands to start/stop/restart/reload units or services:
start: starts a unit if it is not already startedstop: stops a unitrestart: starts or restarts a unitreload: tell a unit to reload its configuration (if it supports it)try-restart: restarts a unit only if it is already active, otherwise do nothing, to prevent accidentally starting a servicereload-or-restart: tell a unit to reload its configuration if supported, otherwise restart ittry-reload-or-restart: tell a unit to reload its configuration if supported, otherwise restart it. If the unit is not already active, do nothing to prevent accidentally starting a service.systemctl has halt, poweroff, reboot, suspend, hibernate, and
hybrid-sleep commands to tell systemd to reboot, power off, suspend and so
on. kexec and switch-root also work.
The rescue and emergency commands switch the system to rescue and emergency
mode (see man systemd.special.
systemctl default switches to the default mode, which also happens when
exiting the rescue or emergency shell.
Run services at boot
systemd does not implement runlevels, and services start at boot based on their
dependencies.
To start a service at boot, you add to its .service file a WantedBy=
dependency on a well-known .target unit.
At boot, systemd brings up the whole chain of dependency started from a default
unit, and that will eventually activate also your service.
See systemctl get-default for what unit is currently the default in your
system. You can change it via the systemd.unit= kernel command line,
so you can configure multiple entries in the boot loader that boot the system
running different services. For example systemd.unit=rescue.target for a
rescue mode, systemd.unit=multi-user.target for a non-graphical mode, or add
your own .target file to implement new system modes.
See systemctl list-units -t target --all for a list of all currently
available targets in your system.
systemctl enable unitname enables the unit to start at boot, by creating
symlinks to it in the .wants directory of the units listed in its
WantedBy= configurationsystemctl disable unitname removes the symlinks created by enablesystemctl reenable unitname removes and readds the symlinks for when you
changed WantedBy=systemctl start activates a unit right now, but does not automatically
enable it at boot
systemctl enable enables a unit at boot, but does not automatically start
it right now
* a disabled unit can still be activated if another unit depends on it
To disable a unit so that it will never get started even if another unit
depends on it, use systemctl mask unitname. Use systemctl unmask unitname
to undo the masking.
Reloading / restarting systemd
systemctl daemon-reload tells systemd to reload its configuration.
systemctl daemon-reexec tells systemd to restart iself.
man systemd.unit),
and each unit is described by a configuration in ini-style format.
For example, this unit continuously plays an alarm sound when the system is in emergency or rescue mode:
[Unit] Description=Beeps when in emergency or rescue mode DefaultDependencies=false StopWhenUnneeded=true [Install] WantedBy=emergency.target rescue.target [Service] Type=simple ExecStart=/bin/sh -ec 'while true; do /usr/bin/aplay -q /tmp/beep.wav; sleep 2; done'
.service: daemons.socket: communication sockets.device: hardware devices.mount: mount points.automount: automounting.swap: swap files or partitions.target: only dependencies, like Debian metapackages.path: inotify monitoring of paths.timer: cron-like activation.slice: group processes for common resource management.scope: group processes for common resource management/lib/systemd/system/: for units provided by packaged software/run/systemd/system/: runtime-generated units/etc/systemd/system/: for units provided by systemd administrators/etc/ override unit files in /lib/. Note that while Debian
uses /lib/, other distributions may use /usr/lib/ instead.
If there is a directory with the same name as the unit file plus a .d suffix,
any file *.conf it contains is parsed after the unit, and can be used to add
or override configuration options.
For example:
/lib/systemd/system/beep.service.d/foo.conf can be used to tweak
the contents of /lib/systemd/system/beep.service, so it is possible for a
package to distribute a tweak to the configuration of another package./etc/systemd/system/beep.service.d/foo.conf can be used to tweak the
contents of /lib/systemd/system/beep.service, so it is possible a system
administrator to extend a packaged unit without needing to replace it
entirely.unitname.wants/ or unitname.requires/ directory can be used to
extend Wants= and Requires= dependencies on other units, by placing
symlinks to other units in them.
See also:
A first update 0.0.2 of the pinp package arrived on CRAN just a few days after the initial release.
We added a new vignette for the package (see below), extended a few nice features, and smoothed a few corners.
Courtesy of CRANberries, there is a comparison to the previous release. More information is on the tint page. For questions or comments use the issue tracker off the GitHub repo.Changes in tint version 0.0.2 (2017-09-20)
- The YAML segment can be used to select font size, one-or-two column mode, one-or-two side mode, linenumbering and watermarks (#21 and #26 addressing #25)
- If
pinp.clsorjss.bstare not present, they are copied in ((#27 addressing #23)- Output is now in shaded framed boxen too (#29 addressing #28)
- Endmatter material is placed in
template.tex(#31 addressing #30)- Expanded documentation of YAML options in
skeleton.Rmdand clarified available one-column option (#32).- Section numbering can now be turned on and off (#34)
- The default bibliography style was changed to
jss.bst.- A short explanatory vignette was added.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
The PATHspider software I maintain as part of my work
depends on some features in cURL and in
PycURL that have
only
just been mereged or are still
awaiting merge. I need to build a
docker container that includes these as Debian packages, so I need to quickly
build an APT repository.
A Debian repository can essentially be seen as a static website and the
contents are GPG signed so it doesn t necessarily need to be hosted somewhere
trusted (unless availability is critical for your application). I host my blog
with Netlify, a static website host, and I figured they
would be perfect for this use case. They also support open source
projects.
There is a CLI tool for netlify which you can install with:
sudo apt install npm
sudo npm install -g netlify-cli
mkdir repository
cp /path/to/*.deb repository/
cd repository
apt-ftparchive packages . > Packages
apt-ftparchive release . > Release
gpg --clearsign -o InRelease Release
netlify deploy
apt-transport-https if you re going
to enable HTTPS though.)
To add this repository to your apt sources:
gpg --export -a YOURKEYID sudo apt-key add -
echo "deb https://SUBDOMAIN.netlify.com/ /" sudo tee -a /etc/apt/sources.list
sudo apt update
__init__.py and remove obsolete earlier code.--variations flag to support parameters to
certain variations like user_group, and document examples in README.--vary flag for the new syntax and deprecate --dont-vary.--extra-build flag.artifact_pattern to avoid arbitrary shell execution.Many voices arise now and then against risks linked to the Github use by Free Software projects. Yet the infatuation for the collaborative forge of the Octocat Californian start-ups doesn t seem to fade away.
These recent years, Github and its services take an important role in software engineering as they are seen as easy to use, efficient for a daily workload with interesting functions in enterprise collaborative workflow or amid a Free Software project. What are the arguments against using its services and are they valid? We will list them first, then we ll examine their validity.
The Github application belongs to a single entity, Github Inc, a US company which manage it alone. So, a unique company under US legislation manages the access to most of Free Software application code sources, which may be a problem with groups using it when a code source is no longer available, for political or technical reason.
The Octocat, the Github mascot
This centralization leads to another trouble: as it obtained critical mass, it becomes more and more difficult not having a Github account. People who don t use Github, by choice or not, are becoming a silent minority. It is now fashionable to use Github, and not doing so is seen as out of date . The same phenomenon is a classic, and even the norm, for proprietary social networks (Facebook, Twitter, Instagram).
When you interact with Github, you are using a proprietary software, with no access to its source code and which may not work the way you think it is. It is a problem at different levels. First, ideologically, but foremost in practice. In the Github case, we send them code we can control outside of their interface. We also send them personal information (profile, Github interactions). And mostly, Github forces any project which goes through the US platform to use a crucial proprietary tools: its bug tracking system.
Windows, the epitome of proprietary software, even if others took the same path
Working with Github interface seems easy and intuitive to most. Lots of companies now use it as a source repository, and many developers leaving a company find the same Github working environment in the next one. This pervasive presence of Github in free software development environment is a part of the uniformization of said developers working space.
Uniforms always bring Army in my mind, here the Clone army
As said above, nowadays, Github is the main repository of Free Software source code. As such it is a favorite target for cyberattacks. DDOS hit it in March and August 2015. On December 15, 2015, an outage led to the inaccessibility of 5% of the repositories. The same occurred on November 15. And these are only the incident reported by Github itself. One can imagine that the mean outage rate of the platform is underestimated.
Today many dependency maintenance tools, as npm for javascript, Bundler for Ruby or even pip for Python can access an application source code directly from Github. Free Software projects getting more and more linked and codependents, if one component is down, all the developing process stop.
One of the best examples is the npmgate. Any company could legally demand that Github take down some source code from its repository, which could create a chain reaction and blocking the development of many Free Software projects, as suffered the Node.js community from the decisions of Npm, Inc, the company managing npm.
Github didn t appear out of the blue. In his time, its predecessor, SourceForge, was also extremely popular.
Heavily centralized, based on strong interaction with the community, SourceForge is now seen as an aging SAAS (Software As A Service) and sees most of its customers fleeing to Github. Which creates lots of hurdles for those who stayed. The Gimp project suffered from spams and terrible advertising, which led to the departure of the VLC project, then from installers corrupted with adwares instead of the official Gimp installer for Windows. And finally, the Project Gimp s SourceForge account was hacked by SourceForge team itself!
These are very recent examples of what can do a commercial entity when it is under its stakeholders pressure. It is vital to really understand what it means to trust them with data and exchange centralization, where it could have tremendous repercussion on the day-to-day life and the habits of the Free Software and open source community.
Mostly based on ideology, this point deals with the definition every member of the community gives to Free Software and open source. Mostly about one thing: is it viral or not? Or GPL vs MIT/BSD.
Those on the side of the viral Free Software will have trouble to use a proprietary software as this last one shouldn t even exist. It must be assimilated, to quote Star Trek, as it is a connected black box, endangering privacy, corrupting for profit our uses and restrain our freedom to use as we re pleased what we own, etc.
Those on the side of complete freedom have no qualms using proprietary software as their very existence is a consequence of freedom without restriction. They even agree that code they developed may be a part of proprietary software, which is quite a common occurrence. This part of the Free Software community has no qualm using Github, which is well within their ideology parameters. Just take a look at the Janson amphitheater during Fosdem and check how many Apple laptops running on macOS are around.
FreeBSD, the main BSD project under the BSD license
Even without ideological consideration, and just focusing on Github infrastructure, the bug tracking system is a major problem by itself.
Bug report builds the memory of Free Software projects. It is the entrance point for new contributors, the place to find bug reporting, requests for new functions, etc. The project history can t be limited only to the code. It s very common to find bug reports when you copy and paste an error message in a search engine. Not their historical importance is precious for the project itself, but also for its present and future users.
Github gives the ability to extract bug reports through its API. What would happen if Github is down or if the platform doesn t support this feature anymore? In my opinion, not that many projects ever thought of this outcome. How could they move all the data generated by Github into a new bug tracking system? One old example now is Astrid, a TODO list bought by Yahoo a few years ago. Very popular, it grew fast until it was closed overnight, with only a few weeks for its users to extract their data. It was only a to-do list. The same situation with Github would be tremendously difficult to manage for several projects if they even have the ability to deal with it. Code would still be available and could still live somewhere else, but the project memory would be lost. A project like Debian has today more than 800,000 bug reports, which are a data treasure trove about problems solved, function requests and where the development stand on each. The developers of the Cpython project have anticipated the problem and decided not to use Github bug tracking systems.
Issues, the Github proprietary bug tracking system
Another thing we could lose if Github suddenly disappear: all the work currently done regarding the push requests (aka PRs). This Github function gives the ability to clone one project s Github repository, to modify it to fit your needs, then to offer your own modification to the original repository. The original repository s owner will then review said modification, and if he or she agrees with them will fuse them into the original repository. As such, it s one of the main advantages of Github, since it can be done easily through its graphic interface.
However reviewing all the PRs may be quite long, and most of the successful projects have several ongoing PRs. And this PRs and/or the proprietary bug tracking system are commonly used as a platform for comment and discussion between developers.
Code itself is not lost if Github is down (except one specific situation as seen below), but the peer review works materialized in the PRs and the bug tracking system is lost. Let s remember than the PR mechanism let you clone and modify projects and then generate PRs directly from its proprietary web interface without downloading a single code line on your computer. In this particular case, if Github is down, all the code and the work in progress is lost. Some also use Github as a bookmark place. They follow their favorite projects activity through the Watch function. This technological watch style of data collection would also be lost if Github is down.
Debian, one of the main Free Software projects with at least a thousand official contributors
The Free Software community is walking a thigh rope between normalization needed for an easier interoperability between its products and an attraction for novelty led by a strong need for differentiation from what is already there.
Github popularized the use of Git, a great tool now used through various sectors far away from its original programming field. Step by step, Git is now so prominent it s almost impossible to even think to another source control manager, even if awesome alternate solutions, unfortunately not as popular, exist as Mercurial.
A new Free Software project is now a Git repository on Github with README.md added as a quick description. All the other solutions are ostracized? How? None or very few potential contributors would notice said projects. It seems very difficult now to encourage potential contributors into learning a new source control manager AND a new forge for every project they want to contribute. Which was a basic requirement a few years ago. It s quite sad because Github, offering an original experience to its users, cut them out of a whole possibility realm. Maybe Github is one of the best web versioning control systems. But being the main one doesn t let room for a new competitor to grow. And it let Github initiate development newcomers into a narrow function set, totally unrelated to the strength of the Git tool itself.
Fight against centralization is a main part of the Free Software ideology as centralization strengthens the power of those who manage it and who through it control those who are managed by it. Uniformization allergies born against main software companies and their wishes to impose a closed commercial software world was for a long time the main fuel for innovation thirst and intelligent alternative development. As we said above, part of the Free Software community was built as a reaction to proprietary software and their threat. The other part, without hoping for their disappearance, still chose a development model opposite to proprietary software, at least in the beginning, as now there s more and more bridges between the two.
The Github effect is a morbid one because of its consequences: at least centralization, uniformization, proprietary software usage as their bug tracking system. But some years ago the Dear Github buzz showed one more side effect, one I ve never thought about: laziness. For those who don t know what it is about, this letter is a complaint from several spokespersons from several Free Software projects which demand to Github team to finally implement, after years of polite asking, new functions. Since when Free Software project facing a roadblock request for clemency and don t build themselves the path they need? When Torvalds was involved in the Bitkeeper problem and the Linux kernel development team couldn t use anymore their revision control software, he developed Git. The mere fact of not being able to use one tool or functions lacking is the main motivation to seek alternative solutions and, as such, of the Free Software movement. Every Free Software community member able to code should have this reflex. You don t like what Github offers? Switch to Gitlab. You don t like it Gitlab? Improve it or make your own solution.
The Gitlab logo
Let s be crystal clear. I ve never said that every Free Software developers blocked should code his or her own alternative. We all have our own priorities, and some of us even like their beauty sleep, including me. But, to see that this open letter to Github has 1340 names attached to it, among them some spokespersons for major Free Software project showed me that need, willpower and strength to code a replacement are here. Maybe said replacement will be born from this letter, it would be the best outcome of this buzz.
In the end, Github usage is just another example of Internet usage massification. As Internet users are bound to go to massively centralized social network as Facebook or Twitter, developers are following the same path with Github. Even if a large fraction of developers realize the threat linked this centralized and proprietary organization, the whole community is following this centralization and uniformization trend. Github service is useful, free or with a reasonable price (depending on the functions you need) easy to use and up most of the time. Why would we try something else? Maybe because others are using us while we are savoring the convenience? The Free Software community seems to be quite sleepy to me.
The lion enjoying the hearth warm
Next.