Out of 46,500,001 electorate 17,410,742 voted to leave, which is a mere 37.4% or just over a third. [source]. On my books this is not a clear expression of the UK wishes.
The reaction that the results have caused are devastating. The Scottish First Minister has announced plans for 2nd Scottish Independence referendum [source]. Londoners are filing petitions calling for Independent London [source, source]. The Prime Minister announced his resignation [source]. Things are not stable.
I do not believe that super majority of the electorate are in favor of leaving the EU. I don't even believe that those who voted to leave have considered the break up of the UK as the inevitable outcome of the leave vote. There are numerous videos on the internet about that, impossible to quantify or reliably cite, but for example this [source]
So What Now?
P R O T E S T
I urge everyone to start protesting the outcome of the mistake that happened last Thursday. 4th of July is a good symbolic date to show your discontent with the UK governemnt and a tiny minority who are about to cause the country to fall apart with no other benefits. Please stand up and make yourself heard.
There are 64,100,000 people living in the UK according to the World Bank, maybe the government should fear and listen to the unheard third. The current "majority" parliament was only elected by 24% of electorate.
It is time for people to actually take control, we can fix our parliament, we can stop austerity, we can prevent the break up of the UK, and we can stay in the EU. Over to you.
ps. How to elect next PM?
Electing next PM will be done within the Conservative Party, and that's kind of a bummer, given that the desperate state the country currently is in. It is not that hard to predict that Boris Johnson is a front-runner. If you wish to elect a different PM, I urge you to splash out 25 quid and register to be a member of the Conservative Party just for one year =) this way you will get a chance to directly elect the new Leader of the Conservative Party and thus the new Prime Minister. You can backdoor the Conservative election here.
Hating jetlag based headache.
Disturbed to see the Brexit result.
Review wiki RecentChanges.
Answer some questions about Launchpad on #debian-mentors.
Whitelisted one user in the wiki anti-spam system.
Reviewed and sponsored yamllint 1.2.2-1 upload.
Noted OFSET repo is broken and updated Freeduc info.
Noted the Epidemic-Linux website is having database issues.
Noted that Facebook finally completely dropped their RSS
feeds, dropped Facebook RSS feed URL generation
from the Debian derivatives census scripts and
notified the affected derivatives.
Cleared up Tanglu hash sum mismatches again.
Minor changes to Planet Debian derivatives.
Enjoyed the photos from Valessio.
Hazy city away from the mountain and tablecloth clouds
flowing over the mountain on the way to a pub lunch.
Jet lag headaches seem to be subsiding thankfully.
Ping someone generating a bounce when changing their SSH key.
Mention autorevision and other suggestions in an IRC discussion
about mesa & reproducible builds.
Review some DebConf16 announcements and add minor fix.
Push out some TODO items to check-all-the-things.
Ask for a dd-list for the GCC 6 transition.
Usual spam reporting throughout the day via manual List-Archive
copy-paste, feeding mboxen to my report-spam-debian-lists and
report-spam-debian-bugs scripts and manual BTS clicks.
Usual wondering why there isn't an RFC for MUA spam reporting.
Disturbed by the sudden appearance of an astronautess in the
orga room but placated by a plentiful supply of crisps.
Ask x32 folks about debian-x32.org vs x32 on ports.d.o.
Glad to just avoid the room shuffle dance.
Finish mime support for check-all-the-things.
Disappointed that piz.za does not actually resolve.
Amused by pollito's virtual tour of UTC.
Completely stuffed full of Butleritos.
I bought some awful WiFi lightbulbs a few months ago. The short version: they introduced terrible vulnerabilities on your network, they violated the GPL and they were also just bad at being lightbulbs. Since then I've bought some other Internet of Things devices, and since people seem to have a bizarre level of fascination with figuring out just what kind of fractal of poor design choices these things frequently embody, I thought I'd oblige.
Today we're going to be talking about the KanKun SP3, a plug that's been around for a while. The idea here is pretty simple - there's lots of devices that you'd like to be able to turn on and off in a programmatic way, and rather than rewiring them the simplest thing to do is just to insert a control device in between the wall and the device andn ow you can turn your foot bath on and off from your phone. Most vendors go further and also allow you to program timers and even provide some sort of remote tunneling protocol so you can turn off your lights from the comfort of somebody else's home.
The KanKun has all of these features and a bunch more, although when I say "features" I kind of mean the opposite. I plugged mine in and followed the install instructions. As is pretty typical, this took the form of the plug bringing up its own Wifi access point, the app on the phone connecting to it and sending configuration data, and the plug then using that data to join your network. Except it didn't work. I connected to the plug's network, gave it my SSID and password and waited. Nothing happened. No useful diagnostic data. Eventually I plugged my phone into my laptop and ran adb logcat, and the Android debug logs told me that the app was trying to modify a network that it hadn't created. Apparently this isn't permitted as of Android 6, but the app was handling this denial by just trying again. I deleted the network from the system settings, restarted the app, and this time the app created the network record and could modify it. It still didn't work, but that's because it let me give it a 5GHz network and it only has a 2.4GHz radio, so one reset later and I finally had it online.
The first thing I normally do to one of these things is run nmap with the -O argument, which gives you an indication of what OS it's running. I didn't really need to in this case, because if I just telnetted to port 22 I got a dropbear ssh banner. Googling turned up the root password ("p9z34c") and I was logged into a lightly hacked (and fairly obsolete) OpenWRT environment.
It turns out that here's a whole community of people playing with these plugs, and it's common for people to install CGI scripts on them so they can turn them on and off via an API. At first this sounds somewhat confusing, because if the phone app can control the plug then there clearly is some kind of API, right? Well ha yeah ok that's a great question and oh good lord do things start getting bad quickly at this point.
I'd grabbed the apk for the app and a copy of jadx, an incredibly useful piece of code that's surprisingly good at turning compiled Android apps into something resembling Java source. I dug through that for a while before figuring out that before packets were being sent, they were being handed off to some sort of encryption code. I couldn't find that in the app, but there was a native ARM library shipped with it. Running strings on that showed functions with names matching the calls in the Java code, so that made sense. There were also references to AES, which explained why when I ran tcpdump I only saw bizarre garbage packets.
But what was surprising was that most of these packets were substantially similar. There were a load that were identical other than a 16-byte chunk in the middle. That plus the fact that every payload length was a multiple of 16 bytes strongly indicated that AES was being used in ECB mode. In ECB mode each plaintext is split up into 16-byte chunks and encrypted with the same key. The same plaintext will always result in the same encrypted output. This implied that the packets were substantially similar and that the encryption key was static.
Some more digging showed that someone had figured out the encryption key last year, and that someone else had written some tools to control the plug without needing to modify it. The protocol is basically ascii and consists mostly of the MAC address of the target device, a password and a command. This is then encrypted and sent to the device's IP address. The device then sends a challenge packet containing a random number. The app has to decrypt this, obtain the random number, create a response, encrypt that and send it before the command takes effect. This avoids the most obvious weakness around using ECB - since the same plaintext always encrypts to the same ciphertext, you could just watch encrypted packets go past and replay them to get the same effect, even if you didn't have the encryption key. Using a random number in a challenge forces you to prove that you actually have the key.
At least, it would do if the numbers were actually random. It turns out that the plug is just calling rand(). Further, it turns out that it never calls srand(). This means that the plug will always generate the same sequence of challenges after a reboot, which means you can still carry out replay attacks if you can reboot the plug. Strong work.
But there was still the question of how the remote control works, since the code on github only worked locally. tcpdumping the traffic from the server and trying to decrypt it in the same way as local packets worked fine, and showed that the only difference was that the packet started "wan" rather than "lan". The server decrypts the packet, looks at the MAC address, re-encrypts it and sends it over the tunnel to the plug that registered with that address.
That's not really a great deal of authentication. The protocol permits a password, but the app doesn't insist on it - some quick playing suggests that about 90% of these devices still use the default password. And the devices are all based on the same wifi module, so the MAC addresses are all in the same range. The process of sending status check packets to the server with every MAC address wouldn't take that long and would tell you how many of these devices are out there. If they're using the default password, that's enough to have full control over them.
There's some other failings. The github repo mentioned earlier includes a script that allows arbitrary command execution - the wifi configuration information is passed to the system() command, so leaving a semicolon in the middle of it will result in your own commands being executed. Thankfully this doesn't seem to be true of the daemon that's listening for the remote control packets, which seems to restrict its use of system() to data entirely under its control. But even if you change the default root password, anyone on your local network can get root on the plug. So that's a thing. It also downloads firmware updates over http and doesn't appear to check signatures on them, so there's the potential for MITM attacks on the plug itself. The remote control server is on AWS unless your timezone is GMT+8, in which case it's in China. Sorry, Western Australia.
It's running Linux and includes Busybox and dnsmasq, so plenty of GPLed code. I emailed the manufacturer asking for a copy and got told that they wouldn't give it to me, which is unsurprising but still disappointing.
The use of AES is still somewhat confusing, given the relatively small amount of security it provides. One thing I've wondered is whether it's not actually intended to provide security at all. The remote servers need to accept connections from anywhere and funnel decent amounts of traffic around from phones to switches. If that weren't restricted in any way, competitors would be able to use existing servers rather than setting up their own. Using AES at least provides a minor obstacle that might encourage them to set up their own server.
Overall: the hardware seems fine, the software is shoddy and the security is terrible. If you have one of these, set a strong password. There's no rate-limiting on the server, so a weak password will be broken pretty quickly. It's also infringing my copyright, so I'd recommend against it on that point alone.
texlive-bin/2016.20160513.41080-3 has been uploaded to unstable,
featuring support for FORCE_SOURCE_DATE. See the last post
for details on it.
doxygen/1.4.4-1 has been uploaded to unstable, fixing #822197
(upstream bug), which caused
some generated html file to contain unreproducible memory addresses of Python
objects used at build time.
debhelper/9.20160618 has been uploaded to unstable, fixing #824490,
which instructs ant to not save the username of the build user in the
generated files. Original patch by Emmanuel Bourg.
epydoc/3.0.1+dfsg-13 and 3.0.1+dfsg-14 have been uploaded by Kenneth J.
Pronovici which fixes nondeterministic ordering issues in generated
documentation and removes memory addresses. Original patches (#825968
and #827416) by Sascha Steinbiss.
With this upload of texlive-bin we decided to stop keeping our patched fork of
as most of the patches for SOURCE_DATE_EPOCH support had been integrated
upstream already, and the last one (making FORCE_SOURCE_DATE default to 1)
had been refused. So, we are now going to let the archive be rebuilt against
unstable's texlive-bin and see how many packages will become unreproducible
with this change; once enough data will be collected we will ponder whether
FORCE_SOURCE_DATE should be exported by helper tools (such as debhelper) or
manually exported by every package that needs it.
(For those wondering: we still recommend to follow SOURCE_DATE_EPOCH always
and don't recommend other projects to implement FORCE_SOURCE_DATE )
With the drop of texlive-bin we now have only three modified packages in our
experimental repository.
Reproducible work in other projects
Ed Maste sent a patch to have
ELF Tool Chain's elfcopy support SOURCE_DATE_EPOCH.
Ed Maste changed
FreeBSD's ar(1) to have a reproducible output by default when invoked with -s.
Ed Maste merged changes
from NetBSD's makefs to FreeBSD's to add a command line option for setting the
timestamp.
Holger Levsen spent some time thinking and documenting how to store package notes
and issues in a multi-distribution friendly manner, so that we can share
these issues and notes between reproducible efforts across different projects.
Thoughts we had about this topic
during the Athens meeting last December are included in the updated README.
Uploads with reproducibility fixes that currently fail to build:
ruby2.3/2.3.1-3 by Christian Hofstaedtler, avoids unreproducible rbconfig.rb files by always using bash for building.
Patches submitted that have not made their way to the archive yet:
#827109 against asciijump by Reiner Herrmann: sort source files for deterministic linking order.
#827112 against boswars by Reiner Herrmann: sort source files for deterministic linking order.
#827114 against overgod by Reiner Herrmann: use C locale for sorting source files.
#827115 against netpbm by Alexis Bienven e: honour SOURCE_DATE_EPOCH while generating output.
#827124 against funguloids by Reiner Herrmann: use C locale for sorting files.
#827145 against scummvm by Reiner Herrmann: don't embed extra fields in zip archive and build with proper host architecture.
#827150 against netpanzer by Reiner Herrmann: sort source files for deterministic linking order.
#827172 against reaver by Alexis Bienven e: sort object files for deterministic linking order.
#827187 against latex2html by Alexis Bienven e: iterate deterministic over Perl hashes; honour SOURCE_DATE_EPOCH for output; strip username from output; sort index keys.
#827313 against cherrypy3 by Sascha Steinbiss: prevent memory addresses in output.
#827361 against matplotlib by Alexis Bienven e: honour SOURCE_DATE_EPOCH in output and sort keys while iterating over dict.
#827382 against dwarfutils by Reiner Herrmann: fix array size, which caused memory from outside a table to be embedded into output.
#827384 against skytools3 by Sascha Steinbiss: use stable sorting order and remove timestamps from documentation.
#827419 against ldaptor by Sascha Steinbiss: sort list of input files and prevent home directory from leaking into documentation.
#827546 against git-buildpackage by Sascha Steinbiss: replace timestamps in documentation with changelog date; prevent temporary paths in documentation.
#827572 against xprobe by Reiner Herrmann: sort list of object files in static library archives.
Package reviews
36 reviews have been added, 12 have been updated and 31 have been removed in this week.
17 FTBFS bugs have been reported by Chris Lamb, Santiago Vila and Dominic Hargreaves.
diffoscope development
Satyam worked on argument completion (#826711) for diffoscope.
strip-nondeterminism development
Mattia Rizzolo uploaded strip-nondeterminism 0.019-1~bpo8+1 to
jessie-backports.
reprotest development
Ceridwen filed an Intent To Package (ITP) bug for reprotest as #827293.
tests.reproducible-builds.org
Mattia Rizzolo uploaded pbuilder 0.225 to unstable, providing built-in
support for eatmydata. We're planning to use it in armhf and i386 builders
where we don't build in tmpfs, to increase the build speed some more.
Valery Young reworked the appearance of the package page, hopefully making them more
intuitive and usable. In the process she changed the script generating them
to use a real templating system, thus improving
maintenance for the future.
Holger adjusted the scheduler to reschedule packages in state 'depwait' after
two days instead of three.
Mattia added the bug title next to the bug numbers in the notes.
Misc.
This week's edition was written by Mattia Rizzolo, Reiner Herrmann, Ed Maste and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.
Jenny Lawson, who blogs as The Bloggess,
is best known for a combination of being extremely funny and being
extremely open about her struggles with mental illness (a combination of
anxiety and depression, alongside a few other problems). Her first book
primarily told the story of her family, childhood, and husband.
Furiously Happy is a more random and disconnected book, but insofar
as it has a theme, it's about surviving depression, anxiety, and other
manifestations of your brain being an asshole.
I described Lawson's previous book, Let's
Pretend This Never Happened, as the closest thing I've found to a
stand-up comedy routine in book form. Furiously Happy is very
similar, but it lacks the cohesiveness of a routine. Instead, it feels
like a blog collection: a pile of essays with only some vague themes and
not a lot of continuity from essay to essay.
This doesn't surprise me. Second books are very different than first
books, particularly second books by someone whose writing focus is not
writing books, and particularly for non-fiction. I feel like Let's
Pretend This Never Happened benefited from drawing on Lawson's full life
experience to form the best book she could write. When that became wildly
popular, everyone of course wanted a second book, including me. But when
the writing is this personal, the second book is, out of necessity, partly
leftovers. Lawson's recent experiences don't generate as much material as
her whole life up to the point of the first book.
That said, there is a bit of a theme, and the title fits it. Early in the
book, Lawson describes how, after the death of a friend and a bout of
depression, she decided to be furiously, vehemently happy to get back at
the universe, to spite its attempts to destroy her mood. It's one of the
best bits in this book. The surrounding philosophy is about embracing the
moment, enjoying the hell out of everything that one can enjoy, and taking
a lot of chances.
A lot of the stories in this book come after the beginning of Lawson's
fame and popularity. She has book tours, a vacation tour of Australia,
and a community of people from her blog. That, of course, doesn't make
the depression and anxiety any better; indeed, it provides a lot of
material for her anxiety to work with. Lawson talks a lot about
surviving, about how important that community is to her, about not
believing your brain when it lies to you. This isn't as uniformly funny
as her first book, and sometimes it feels a bit too much like an earnest
pep talk. But there are also moments of insightful life philosophy mixed
into the madcap adventures and stream-of-consciousness wild associations.
Lawson also does for anxiety what Allie Brosh
does for depression: make the severe form of it relatable to people who
have not suffered from it. I was particularly struck by her description
of flying: the people around her are getting nervous and anxious as the
plane starts to take off, and she's finally able to relax because her
anxiety focused on all the things she had to do in order to get onto the
right plane at the right time. Once she didn't have to make any decisions
or do anything other than sit in one place, her anxiety let go. I don't
have any type of clinical anxiety, but I was able to identify with that
moment of relief and its contrast with anxiety in a deeper way than with
other descriptions.
Furiously Happy is a bit more serious and earnest, and I'm not sure
it worked as well. I liked Lawson's first book better; this felt more
like a blog archive. But she's still funny, entertaining, and delightful,
and I'm happy to support her with a book purchase. Start with either
her blog or Let's Pretend This Never
Happened if you're new to Lawson, but if you're already a fan, here's more
of her writing.
Rating: 7 out of 10
I'm back from the GTK hackfest in Toronto, Canada and mostly recovered from
jetlag, so it's time to write up my notes on what we discussed there.
Despite the hackfest's title, I was mainly there to talk about non-GUI
parts of the stack, and technologies that fit more closely in what could
be seen as the freedesktop.org platform than they do in GNOME. In particular,
I'm interested in Flatpak as a way to deploy self-contained "apps" in a
freedesktop-based, sandboxed runtime environment layered over
the Universal Operating System and
its many derivatives,
with both binary and source compatibility with other GNU/Linux distributions.
I'm mainly only writing about discussions I was directly involved in:
lots of what sounded like good discussion about the actual graphics toolkit
went over my head completely :-)
More notes, mostly from Matthias
Clasen, are available on the GNOME wiki.
In no particular order:
Thinking with portals
We spent some time discussing Flatpak's portals, mostly on Tuesday.
These are the components that expose a subset of desktop functionality
as D-Bus services that can be used by contained applications: they are
part of the security boundary between a contained app and the rest of the
desktop session. Android's intents are a similar concept seen elsewhere.
While the portals are primarily designed for Flatpak, there's no real
reason why they couldn't be used by other app-containment solutions
such as Canonical's Snap.
One major topic of discussion was their overall design and layout. Most
portals will consist of a UX-independent part in Flatpak itself, together
with a UX-specific implementation of any user interaction the portal
needs. For example, the portal for file selection has a D-Bus service
in Flatpak, which interacts with some UX-specific service that will
pop up a standard UX-specific "Open" dialog for GNOME and probably
other GTK environments, that dialog is in (a branch of) GTK.
A design principle that was reiterated in this discussion is that
the UX-independent part should do as much as possible, with the
UX-specific part only carrying out the user interactions that need
to comply with a particular UX design (in the GTK case, GNOME's design).
This minimizes the amount of work that needs to be redone for other
desktop or embedded environments, while still ensuring that the other
environments can have their chosen UX design. In particular, it's
important that, as much as possible, the security- and performance-sensitive
work (such as data transport and authentication) is shared between all
environments.
The aim is for portals to get the user's permission to carry out actions,
while keeping it as implicit as possible, avoiding an "are you sure?" step
where feasible. For example, if an application asks to open a file,
the user's permission is implicitly given by them selecting the file
in the file-chooser dialog and pressing OK: if they do not want this
application to open a file at all, they can deny permission by cancelling.
Similarly, if an application asks to stream webcam data, the UX we expect
is for GNOME's Cheese app (or a similar non-GNOME app) to appear, open
the webcam to provide a preview window so they can see what they are about
to send, but not actually start sending the
stream to the requesting app until the user has pressed a "Start" button.
When defining the API "contracts" to be provided by applications in that
situation, we will need to be clear about whether the provider is expected
to obtain confirmation like this: in most cases I would anticipate that
it is.
One security trade-off here is that we have to have a small amount of
trust in the providing app. For example, continuing the example of Cheese
as a webcam provider, Cheese could (and perhaps should) be a contained
app itself, whether via something like Flatpak, an LSM like AppArmor
or both. If Cheese is compromised somehow, then whenever it is running, it
would be technically possible for it to open the webcam, stream video and
send it to a hostile third-party application. We concluded that this is an
acceptable trade-off: each application needs to be trusted with the privileges
that it needs to do its job, and we should not put up barriers that are easy
to circumvent or otherwise serve no purpose.
The main (only?) portal so far is the file chooser, in which the contained
application asks the wider system to show an "Open..." dialog, and if the user
selects a file, it is returned to the contained application through a FUSE
filesystem, the document portal. The reference implementation of the UX for
this is in GTK, and is basically a GtkFileChooserDialog. The intention is
that other environments such as KDE will substitute their own equivalent.
Other planned portals include:
image capture (scanner/camera)
opening a specified URI
this needs design feedback on how it should work for non-http(s)
sharing content, for example on social networks (like Android's Sharing menu)
proxying joystick/gamepad input (perhaps via Wayland or FUSE, or perhaps
by modifying libraries like SDL with a new input source)
network proxies (GProxyResolver) and availability (GNetworkMonitor)
contacts/address book, probably vCard-based
notifications, probably based on freedesktop.org Notifications
video streaming (perhaps using Pinot, analogous to PulseAudio but for video)
Environment variables
GNOME on Wayland currently has a problem with environment variables:
there are some traditional ways to set environment variables for X11
sessions or login shells using shell script fragments (/etc/X11/Xsession.d,
/etc/X11/xinit/xinitrc.d, /etc/profile.d), but these do not apply to
Wayland, or to noninteractive login environments like cron and
systemd --user. We are also keen to avoid requiring a Turing-complete shell
language during session startup, because it's difficult to reason about
and potentially rather inefficient.
Some uses of environment variables can be dismissed as unnecessary or even
unwanted, similar to the statement in Debian Policy 9.9: "A program must not
depend on environment variables to get reasonable defaults." However,
there are two common situations where environment variables can be necessary
for proper OS integration: search-paths like $PATH, $XDG_DATA_DIRS and
$PYTHONPATH (particularly necessary for things like Flatpak), and
optionally-loaded modules like $GTK_MODULES and $QT_ACCESSIBILITY
where a package influences the configuration of another package.
There is a stopgap solution in GNOME's gdm display manager,
/usr/share/gdm/env.d, but this is gdm-specific and
insufficiently expressive to provide the functionality needed by
Flatpak: "set XDG_DATA_DIRS to its specified default value if unset,
then add a couple of extra paths".
pam_env comes closer PAM is run at every transition from "no user logged
in" to "user can execute arbitrary code as themselves" but it doesn't
support .d fragments, which are required if we want distribution packages
to be able to extend search paths. pam_env also turns off per-user
configuration by default, citing security concerns.
I'll write more about this when I have a concrete proposal for how to solve it.
I think the best solution is probably a PAM module similar to pam_env but
supporting .d directories, either by modifying pam_env directly or
out-of-tree, combined with clarifying what the security concerns for
per-user configuration are and how they can be avoided.
Relocatable binary packages
On Windows and OS X, various GLib APIs automatically discover where the
application binary is located and use search paths relative to that;
for example, if C:\myprefix\bin\app.exe is running, GLib might put
C:\myprefix\share into the result of g_get_system_data_dirs(),
so that the application can ask to load app/data.xml from the data
directories and get C:\myprefix\share\app\data.xml. We would like
to be able to do the same on Linux, for example so that the apps in a
Flatpak or Snap package can be constructed from RPM or dpkg packages without
needing to be recompiled for a different --prefix, and so that other
third-party software packages like the games on Steam and gog.com can
easily locate their own resources.
Relatedly, there are currently no well-defined semantics for what happens
when a .desktop file or a D-Bus .service file has Exec=./bin/foo.
The meaning of Exec=foo is well-defined (it searches $PATH) and the
meaning of Exec=/opt/whatever/bin/foo is obvious. When this came up in
D-Bus previously, my assertion was that the meaning should be the same as
in .desktop files, whatever that is.
We agreed to propose that the meaning of a non-absolute path in a .desktop
or .service file should be interpreted relative to the directory
where the .desktop or .service file was found: for example, if
/opt/whatever/share/applications/foo.desktop says Exec=../../bin/foo,
then /opt/whatever/bin/foo would be the right thing to execute.
While preparing a mail to the freedesktop and D-Bus mailing lists proposing
this, I found that I had
proposed the same thingalmost 2 years ago...
this time I hope I can actually make it happen!
Flatpak and OSTree bug fixing
On the way to the hackfest, and while the discussion moved to topics that I
didn't have useful input on, I spent some time fixing up the Debian packaging
for Flatpak and its dependencies. In particular, I did my first upload
as a co-maintainer of bubblewrap,
uploaded ostree to unstable (with the known limitation that the grub, dracut
and systemd integration is missing for now
since I haven't been able to test it yet), got
most of the way through packaging Flatpak 0.6.5 (which I'll upload soon),
cherry-picked the right patches to make ostree compile on Debian 8 in an effort
to make backports trivial, and spent some time disentangling
a flatpak test failure
which was breaking the Debian package's installed-tests.
I'm still looking into
ostree test failures on little-endian MIPS,
which I was able to reproduce on a Debian porterbox just before the end of the
hackfest.
OSTree + Debian
I also had some useful conversations with developers from
Endless, who recently opened up a version of their
OSTree build scripts
for public access. Hopefully that information brings me a bit closer to
being able to publish a walkthrough for
how to deploy a simple Debian derivative using OSTree
(help with that is very welcome of course!).
GTK life-cycle and versioning
The life-cycle of GTK releases has already been
mentioned here and elsewhere, and there are some interesting responses
in the comments on my earlier blog post.
It's important to note that what we
discussed at the hackfest is only a proposal: a hackfest discussion between
a subset of the GTK maintainers and a small number of other GTK users
(I am in the latter category)
doesn't, and shouldn't, set policy for all of GTK or for all of GNOME. I
believe the intention is that the GTK maintainers will discuss the proposals
further at GUADEC, and make a decision after that.
As I said before, I hope that being more realistic about API and ABI
guarantees can avoid GTK going too far towards either of the possible
extremes: either becoming unable to advance because it's too constrained
by compatibility, or breaking applications because it isn't constrained
enough. The current situation, where it is meant to be compatible within
the GTK 3 branch but in practice applications still sometimes break,
doesn't seem ideal for anyone, and I hope we can do better in future.
Acknowledgements
Thanks to everyone involved, particularly:
Matthias Clasen, who organised the hackfest and took a lot of notes
Allison Lortie, who provided on-site cat-herding and led us to some
excellent restaurants
Red Hat Inc., who provided the venue (a conference room
in their Toronto office), snacks, a lot of coffee, and several participants
my employers Collabora Ltd., who sponsored my travel and accomodation
After last week's bike ride, I had to pack my bags, and get on Monday
morning on a plane to Seattle. The time was so short that I even left
the bike mounted on the car.
So 8:15, plane to Frankfurt, and then plane to Seattle. To my big
surprise, the Lufthansa "extended leg room" seats were overly
generous; I could actually extend my foot completely and put it on the
back of the seat in front of me. Very good value when travelling in
economy The only downside was that these were "standard" not
"premium" economy, so the seat had leg room but was very narrow. And
with normal sized adults on either side of me, it was somewhat
difficult. The leg space allowed me to work on my laptop without
fearing the person in front of me will recline their seat and break my
screen (almost happened once).
The funniest thing when travelling is that food is always tricky: even
familiar food can be not what you expect. Case in point, me at the
salad buffet, seeing slices of green vegetables, and asking myself:
Are those jalapeno slices, or bell pepper slices? Hmm, I'm sure
they're bell pepper , which resulted in my first vegetable salad with
jalapeno. Would definitely recommend if you like spicy things!
Otherwise the trip was as usual, but shorter and more densely packed
with meetings with a one day exception: had the opportunity to
experience for the first time Whirlyball, which was
more fun and more difficult than it first looked. Also spent an
afternoon on Whidbey Island, which fortunately was also
the nicest day, weather-wise, of the week (all phone pictures, not
colour corrected, straight-out-of-phone):
A few more pictures
here. I
keep being amazed by the nature in this area, definitely my preferred
place in US from the relatively few I visited.
Had a nice dinner as well at Cafe Langley,
which surprisingly had reasonably-authentic Mediterranean food; the
Baba Ganoush was excellent.
Other than that outing, nothing worth describing, except that I really
missed my Zwift or outdoor rides. The experience of using the
stationary bikes in the hotel does not compare, so I resorted more to
running on the treadmill (hmm, Zwift for running, hmm ); if the foot
pod calibration is to be trusted, I continued to slightly improve my
1K, 1mi, and 5K times. Not bad, I might want to join some running
races as well this summer, but I need to take it easy though and make
sure to not get injured again.
And finally, week over, flew back home, slept a bit mid-day (which
will ruin my jet lag recovery program), unloaded my bike from the car
(and checked it still works), and andand got on the trainer and
did a Zwift ride. Jet lagged, but managed to beat my Watopia sprint
record by a tiny bit, and complete a new workout ( The Gorby ), which
was interesting. I can stop any time I want, definitely (I just need
to take a trip away from home ).
I came across Debian Backports recently after a kind soul suggested it in my Jessie Installation article. Backports provide the bridge between two stable releases of Debian. Say you are running Jessie stable in your system now. While it runs great, Jessie stable will be a bit outdated compared to the testing release. This is solved by Backports which recompiles and adjusts these testing packages for the stable environment. Backports can be included into the source file next to the stable sources unlike the testing sources which can misbehave and have a significant risk, Backports are much more compatible and lower risk.
However, Backports cannot be tested as extensively as Debian stable, and backports are provided on an as-is basis, with risk of incompatibilities with other components in Debian stable. Use with care! It is therefore recommended to only select single backported packages that fit your needs, and not use all available backports.
To add Backports to your system,
Open the sources.list file : sudo nano /etc/apt/sources.list
For jessie add the line : deb http://ftp.debian.org/debian jessie-backports main
For wheezy add the line : deb http://ftp.debian.org/debian wheezy-backports main
After any change be sure to run sudo apt-get update
To know more about Backports and to contribute consult the main page here and the wiki here. Happy hacking!
As some of the world knows full well by now, I've been noodling with Go
for a few years, working through its pros, its cons, and thinking a lot
about how humans use code to express thoughts and ideas. Go's got a lot of
neat use cases, suited to particular problems, and used in the right place,
you can see some clear massive wins.
I've started writing Debian tooling in Go, because it's a pretty natural fit.
Go's fairly tight, and overhead shouldn't be taken up by your operating system.
After a while, I wound up hitting the usual blockers, and started to build up
abstractions. They became pretty darn useful, so, this blog post is announcing
(a still incomplete, year old and perhaps API changing) Debian package for Go.
The Go importable name is pault.ag/go/debian. This contains a lot of utilities
for dealing with Debian packages, and will become an edited down "toolbelt"
for working with or on Debian packages.
Module Overview
Currently, the package contains 4 major sub packages. They're a changelog
parser, a control file parser, deb file format parser, dependency parser
and a version parser. Together, these are a set of powerful building blocks
which can be used together to create higher order systems with reliable
understandings of the world.
changelog
The first (and perhaps most incomplete and least tested) is a changelog file
parser.. This provides the
programmer with the ability to pull out the suite being targeted in the
changelog, when each upload was, and the version for each. For example, let's
look at how we can pull when all the uploads of Docker to sid took place:
funcmain()resp,err:=http.Get("http://metadata.ftp-master.debian.org/changelogs/main/d/docker.io/unstable_changelog")iferr!=nilpanic(err)allEntries,err:=changelog.Parse(resp.Body)iferr!=nilpanic(err)for_,entry:=rangeallEntriesfmt.Printf("Version %s was uploaded on %s\n",entry.Version,entry.When)
The output of which looks like:
Version 1.8.3~ds1-2 was uploaded on 2015-11-04 00:09:02 -0800 -0800
Version 1.8.3~ds1-1 was uploaded on 2015-10-29 19:40:51 -0700 -0700
Version 1.8.2~ds1-2 was uploaded on 2015-10-29 07:23:10 -0700 -0700
Version 1.8.2~ds1-1 was uploaded on 2015-10-28 14:21:00 -0700 -0700
Version 1.7.1~dfsg1-1 was uploaded on 2015-08-26 10:13:48 -0700 -0700
Version 1.6.2~dfsg1-2 was uploaded on 2015-07-01 07:45:19 -0600 -0600
Version 1.6.2~dfsg1-1 was uploaded on 2015-05-21 00:47:43 -0600 -0600
Version 1.6.1+dfsg1-2 was uploaded on 2015-05-10 13:02:54 -0400 EDT
Version 1.6.1+dfsg1-1 was uploaded on 2015-05-08 17:57:10 -0600 -0600
Version 1.6.0+dfsg1-1 was uploaded on 2015-05-05 15:10:49 -0600 -0600
Version 1.6.0+dfsg1-1~exp1 was uploaded on 2015-04-16 18:00:21 -0600 -0600
Version 1.6.0~rc7~dfsg1-1~exp1 was uploaded on 2015-04-15 19:35:46 -0600 -0600
Version 1.6.0~rc4~dfsg1-1 was uploaded on 2015-04-06 17:11:33 -0600 -0600
Version 1.5.0~dfsg1-1 was uploaded on 2015-03-10 22:58:49 -0600 -0600
Version 1.3.3~dfsg1-2 was uploaded on 2015-01-03 00:11:47 -0700 -0700
Version 1.3.3~dfsg1-1 was uploaded on 2014-12-18 21:54:12 -0700 -0700
Version 1.3.2~dfsg1-1 was uploaded on 2014-11-24 19:14:28 -0500 EST
Version 1.3.1~dfsg1-2 was uploaded on 2014-11-07 13:11:34 -0700 -0700
Version 1.3.1~dfsg1-1 was uploaded on 2014-11-03 08:26:29 -0700 -0700
Version 1.3.0~dfsg1-1 was uploaded on 2014-10-17 00:56:07 -0600 -0600
Version 1.2.0~dfsg1-2 was uploaded on 2014-10-09 00:08:11 +0000 +0000
Version 1.2.0~dfsg1-1 was uploaded on 2014-09-13 11:43:17 -0600 -0600
Version 1.0.0~dfsg1-1 was uploaded on 2014-06-13 21:04:53 -0400 EDT
Version 0.11.1~dfsg1-1 was uploaded on 2014-05-09 17:30:45 -0400 EDT
Version 0.9.1~dfsg1-2 was uploaded on 2014-04-08 23:19:08 -0400 EDT
Version 0.9.1~dfsg1-1 was uploaded on 2014-04-03 21:38:30 -0400 EDT
Version 0.9.0+dfsg1-1 was uploaded on 2014-03-11 22:24:31 -0400 EDT
Version 0.8.1+dfsg1-1 was uploaded on 2014-02-25 20:56:31 -0500 EST
Version 0.8.0+dfsg1-2 was uploaded on 2014-02-15 17:51:58 -0500 EST
Version 0.8.0+dfsg1-1 was uploaded on 2014-02-10 20:41:10 -0500 EST
Version 0.7.6+dfsg1-1 was uploaded on 2014-01-22 22:50:47 -0500 EST
Version 0.7.1+dfsg1-1 was uploaded on 2014-01-15 20:22:34 -0500 EST
Version 0.6.7+dfsg1-3 was uploaded on 2014-01-09 20:10:20 -0500 EST
Version 0.6.7+dfsg1-2 was uploaded on 2014-01-08 19:14:02 -0500 EST
Version 0.6.7+dfsg1-1 was uploaded on 2014-01-07 21:06:10 -0500 EST
control
Next is one of the most complex, and one of the oldest parts of go-debian,
which is the control file parser
(otherwise sometimes known as deb822). This module was inspired by the way
that the json module works in Go, allowing for files to be defined in code
with a struct. This tends to be a bit more declarative, but also winds up
putting logic into struct tags, which can be a nasty anti-pattern if used too
much.
The first primitive in this module is the concept of a Paragraph, a struct
containing two values, the order of keys seen, and a map of string to string.
All higher order functions dealing with control files will go through this
type, which is a helpful interchange format to be aware of. All parsing of
meaning from the Control file happens when the Paragraph is unpacked into
a struct using reflection.
The idea behind this strategy that you define your struct, and let the Control
parser handle unpacking the data from the IO into your container, letting you
maintain type safety, since you never have to read and cast, the conversion
will handle this, and return an Unmarshaling error in the event of failure.
Additionally, Structs that define an anonymous member of control.Paragraph
will have the raw Paragraph struct of the underlying file, allowing the
programmer to handle dynamic tags (such as X-Foo), or at least, letting
them survive the round-trip through go.
The default decoder
contains an argument, the ability to verify the input control file using an
OpenPGP keyring, which is exposed to the programmer through the
(*Decoder).Signer() function. If the passed argument is nil, it will not
check the input file signature (at all!), and if it has been passed, any
signed data must be found or an error will fall out of the NewDecoder call.
On the way out, the opposite happens, where the struct is introspected,
turned into a control.Paragraph, and then written out to the io.Writer.
Here's a quick (and VERY dirty) example showing the basics of reading and
writing Debian Control files with go-debian.
packagemainimport("fmt""io""net/http""strings""pault.ag/go/debian/control")typeAllowedPackagestructPackagestringFingerprintstringfunc(a*AllowedPackage)UnmarshalControl(instring)errorin=strings.TrimSpace(in)chunks:=strings.SplitN(in," ",2)iflen(chunks)!=2returnfmt.Errorf("Syntax sucks: '%s'",in)a.Package=chunks[0]a.Fingerprint=chunks[1][1:len(chunks[1])-1]returnniltypeDMUAstructFingerprintstringUidstringAllowedPackages[]AllowedPackage control:"Allow" delim:"," funcmain()resp,err:=http.Get("http://metadata.ftp-master.debian.org/dm.txt")iferr!=nilpanic(err)decoder,err:=control.NewDecoder(resp.Body,nil)iferr!=nilpanic(err)fordmua:=DMUAiferr:=decoder.Decode(&dmua);err!=niliferr==io.EOFbreakpanic(err)fmt.Printf("The DM %s is allowed to upload:\n",dmua.Uid)for_,allowedPackage:=rangedmua.AllowedPackagesfmt.Printf(" %s [granted by %s]\n",allowedPackage.Package,allowedPackage.Fingerprint)
Output (truncated!) looks a bit like:
...
The DM Allison Randal <allison@lohutok.net> is allowed to upload:
parrot [granted by A4F455C3414B10563FCC9244AFA51BD6CDE573CB]
...
The DM Benjamin Barenblat <bbaren@mit.edu> is allowed to upload:
boogie [granted by 3224C4469D7DF8F3D6F41A02BBC756DDBE595F6B]
dafny [granted by 3224C4469D7DF8F3D6F41A02BBC756DDBE595F6B]
transmission-remote-gtk [granted by 3224C4469D7DF8F3D6F41A02BBC756DDBE595F6B]
urweb [granted by 3224C4469D7DF8F3D6F41A02BBC756DDBE595F6B]
...
The DM <aelmahmoudy@sabily.org> is allowed to upload:
covered [granted by 41352A3B4726ACC590940097F0A98A4C4CD6E3D2]
dico [granted by 6ADD5093AC6D1072C9129000B1CCD97290267086]
drawtiming [granted by 41352A3B4726ACC590940097F0A98A4C4CD6E3D2]
fonts-hosny-amiri [granted by BD838A2BAAF9E3408BD9646833BE1A0A8C2ED8FF]
...
...
deb
Next up, we've got the deb module. This contains code to handle reading
Debian 2.0 .deb files. It contains a wrapper that will parse the control
member, and provide the data member through the
archive/tar interface.
Here's an example of how to read a .deb file, access some metadata, and
iterate over the tar archive, and print the filenames of each of the
entries.
dependency
The dependency package provides an interface to parse and compute
dependencies. This package is a bit odd in that, well, there's no other
library that does this. The issue is that there are actually two different
parsers that compute our Dependency lines, one in Perl (as part of dpkg-dev)
and another in C (in dpkg).
To date, this has resulted in me filing
threedifferentbugs.
I also found a broken package in the
archive,
which actually resulted in another bug being (totally accidentally)
already fixed.
I hope to continue to run the archive through my parser in hopes of finding
more bugs! This package is a bit complex, but it basically just returns what
amounts to be an AST
for our Dependency lines. I'm positive there are bugs, so file them!
funcmain()dep,err:=dependency.Parse("foo bar, baz, foobar [amd64] bazfoo [!sparc], fnord:armhf [gnu-linux-sparc]")iferr!=nilpanic(err)anySparc,err:=dependency.ParseArch("sparc")iferr!=nilpanic(err)for_,possi:=rangedep.GetPossibilities(*anySparc)fmt.Printf("%s (%s)\n",possi.Name,possi.Arch)
Gives the output:
foo (<nil>)
baz (<nil>)
fnord (armhf)
version
Right off the bat, I'd like to thank
Michael Stapelberg for letting me graft this
out of dcs and into the go-debian package.
This was nearly entirely his work (with a one or two line function I added
later), and was amazingly helpful to have. Thank you!
This module implements Debian version comparisons and parsing, allowing for
sorting in lists, checking to see if it's native or not, and letting the
programmer to implement smart(er!) logic based on upstream (or Debian)
version numbers.
This module is extremely easy to use and very straightforward, and not worth
writing an example for.
Final thoughts
This is more of a "Yeah, OK, this has been useful enough to me at this point
that I'm going to support this" rather than a "It's stable!" or even
"It's alive!" post. Hopefully folks can report bugs and help iterate on
this module until we have some really clean building blocks to build
solid higher level systems on top of. Being able to have multiple libraries
interoperate by relying on go-debian will be a massive ease.
I'm in need of more documentation, and to finalize some parts of the older
sub package APIs, but I'm hoping to be at a "1.0" real soon now.
Wheezy's LTS period started a few weeks ago and the LTS team had to make an
early support decision concerning the Java eco-system since Wheezy ships two Java
runtime environments OpenJDK 6 and
OpenJDK 7. (To be fair, there are
actually three but gcj has
been superseded by OpenJDK a long time ago and the latter should be preferred
whenever possible.)
OpenJDK 6 is currently maintained by Red Hat and we mostly rely on their
upstream work as well as on package updates from Debian's maintainer Matthias Klose and
Tiago St rmer Daitx from Ubuntu. We already knew that both intend
to support OpenJDK 6 until April 2017 when Ubuntu 12.04 will reach its
end-of-life. Thus we had basically two options, supporting OpenJDK 6 for another
twelve months or dropping support right from the start. One of my first steps
was to ask for feedback and advice on
debian-java since
supporting only one JDK seemed to be the more reasonable solution. We agreed on warning users
via various channels about the intended change, especially about possible
incompatibilities
with OpenJDK 7. Even Andrew Haley, OpenJDK 6 project lead, participated in the discussion and confirmed that, while still supported, OpenJDK 6 security
releases are "always the last in the queue when there is urgent work to be
done".
I informed debian-lts about
my findings and issued a call for tests later.
Eventually we decided to concentrate our efforts on OpenJDK
7 because we are confident that for the
majority of our users one Java implementation is sufficient during a stable
release cycle. An immediate positive effect in making OpenJDK 7 the default is that
resources can be relocated to more pressing issues. On the other hand we were
also forced to make compromises. The switch to a newer default implementation
usually triggers a major transition with dozens of FTBFS bugs and the OpenJDK 7
transition
was no exception. I pondered about the usefulness of fixing all these bugs for
Wheezy LTS again and focussing on runtime issues instead and finally decided
that the latter was both more reasonable and more economic.
Different from regular default Java changes, users will still be able to use OpenJDK 6 to
compile their packages and the security impact for development systems is in general neglectable.
More important was to avoid runtime installations of OpenJDK 6. I identified eighteen
packages that strictly depended on the now obsolete JRE and fixed those
issues
on 4 May 2016 together with an update of
java-common and announced the switch to
OpenJDK 7 with a Debian NEWS file.
If you are not a regular reader of Debian
news and also not subscribed to
debian-lts,
debian-lts-announce or
debian-java, remember 26 June
2016 is the day when OpenJDK 7 will be made the default Java implementation in
Wheezy LTS. Of course there is no need to wait. You can switch right now:
Hi to all the souls on planet.debian.org . I hope to meet many of you in Debconf16 which is being held at UCT (University of Cape Town), Rhondenbosch, South Africa and am excited to be part of it. I pushed a blog post about my journey to debconf till date.
I express my sympathy and condolences to all the people who died in the cowardly shooting spree done by a madman in Orlando . I have been upset about this development but as what s done is done, it s best to just keep moving.
Closer to my own reality, I was shocked to discover during my whole visa experience that nowhere there is any knowledge about vaccinations that people should have when they are traveling internationally. I was kinda rudely awakened by this mail which prompted me to take Hep A shot and also start a thread on the mailing list. The more comprehensive info. I got was at the CDC site . It seems to be a go-to site to find about what is recommended. While I will not be taking any shots now as it s nearing to the date of travel, if I had known before, it would have been a valuable resource in itself. Definitely something to be bookmarked if you are going overseas and are worried or have health concerns.
On another note, About few days back, there was a discussion as recently, travel advisories have been issued about possible terror action in South Africa by various embassies due to the holy month of Ramadan.
And the recent attack just proves that it takes just one twisted personality with a perverse sense of justice or whatever s/he/ thinks as just to do what s/he did and guns and more security and not the answer.
My take on it some would describe as simplistic, there are things which you can control, there are things beyond your control. No security agency, no country can guarantee it. By being either home or away, you can t wish you will not get bitten. Got images of Final Destination invoked when I was sharing that, for as an Indian, do believe a bit on fate and karma. Also, me being single also plays a part, perhaps I would have been more cautious or have different motivations if I had a family so I do understand some of the concerns which have been raised by people in that thread. At the end of it, it really is a non-choice in my book.
If you don t take part due to fear,uncertainty of a possible attack, you have already given in to fear and uncertainty and I believe this goes against the very philosophy of what Debian stands for, being bold, taking chances and having trust in your fellow men. If we haven t allowed proprietary, commercial software to win over us, how can we allow less than 0.1% radically motivated people to scare us ? And the recent attack just proves that it takes just one twisted personality with a perverse sense of justice or whatever s/he/ thinks as just to do what s/he did. More guns and more security are certainly not the answer here.
A more troubling part is not terrorism but caste-ism and racism which have been also making news (not in a good way) in India. Now while I cannot claim to have any knowledge about Africans apart from 2-3 conversations which didn t go anywhere, two-three preconceived notions about them can easily be countered. As far as drugs are concerned, IF some Africans are doing drugs smuggling, it would be wrong to pin all of it onto them. India has been fighting drugs smuggling from the 70 s itself. From what we know and have learnt over the years from consuming media, India shares porous borders with almost all our neighbors. Of those, Pakistan is supposed to be the largest grower and supplier and then Nepal where young boys are used as Traffickers. The recent film Rocky Handsome and the recent upcoming controversial movie Udta Punjab are trying to explore these issues. As far as drinking in open is concerned, I have seen and been part of Punjabi parties where both men and women drink without abandon in farm parties. Russians, Germans and Israelis can also out-drink a person on their day/mood. As far as sexuality is concerned, we are the second most populous country in the world, so the less said the better .
I believe though that underneath this racism is money, greed, phobia, language barriers and just maybe some lifestyle choices as well. I have seen North-Eastern, Chinese, Buddhist people being colored with the same brush. The more interesting case is with the Buddhist as can be seen in Himachal Pradesh, where locals feel they are deprived as Europeans come and indulge Buddhists for their monasteries and their way of life while locals don t get much money from them. This is partly true, but also due to our own short-comings in dealing with westerners. I have never seen my own countrymen going out of their way to make westerners or any tourists for that matter feel welcome in a genuine way. More often, the behavior is between hostility, jealousy and a perverted sense of hero-worship due to the color of skin. As far as being racist and bigoted are concerned, it seems we are not alone, just day before came across this news from Malaysia which is interesting in the sense that how people from different communities frame their own history and people around them, forget that is false and misleading to what we in India know to be the truth, it probably is/would be interesting for somebody who does comparative history, myth and folklore analysis. I was actually planning to talk about some of the talks I am looking forward to hearing and seeing on Debconf but guess that will have to wait for few more days. Hope to publish one more before actually flying to Debconf. Till later. Filed under: Miscellenous Tagged: #Debconf16, #racism, #terrorism, #vaccinations
The last few weeks have been pretty hectic in their way, but I ve also had the chance to take some time off work to spend with family, which has been nice.
Memorial Day: breakfast and mud
For Memorial Day, I decided it would be nice to have a cookout for breakfast rather than for dinner. So we all went out to the fire ring. Jacob and Oliver helped gather kindling for the fire, while Laura chopped up some vegetables. Once we got a good fire going, I cooked some scrambled eggs in a cast iron skillet, mixed with meat and veggies. Mmm, that was tasty.
Then we all just lingered outside. Jacob and Oliver enjoyed playing with the cats, and the swingset, and then . water. They put the hose over the slide and made a water slide (more mud slide maybe).
Then we got out the water balloon fillers they had gotten recently, and they loved filling up water balloons. All in all, we all just enjoyed the outdoors for hours.
Flying to Petit Jean, Arkansas
Somehow, neither Laura nor I have ever really been to Arkansas. We figured it was about time. I had heard wonderful things about Petit Jean State Park from other pilots: it s rather unique in that it has a small airport right in the park, a feature left over from when Winthrop Rockefeller owned much of the mountain.
And what a beautiful place it was! Dense forests with wonderful hiking trails, dotted with small streams, bubbling springs, and waterfalls all over; a nice lake, and a beautiful lodge to boot. Here was our view down into the valley at breakfast in the lodge one morning:
And here s a view of one of the trails:
The sunset views were pretty nice, too:
And finally, the plane we flew out in, parked all by itself on the ramp:
It was truly a relaxing, peaceful, re-invigorating place.
Flying to Atchison
Last weekend, Laura and I decided to fly to Atchison, KS. Atchison is one of the oldest cities in Kansas, and has quite a bit of history to show off. It was fun landing at the Amelia Earhart Memorial Airport in a little Cessna, and then going to three museums and finding lunch too.
Of course, there is the Amelia Earhart Birthplace Museum, which is a beautifully-maintained old house along the banks of the Missouri River.
I was amused to find this hanging in the county historical society museum:
One fascinating find is a Regina Music Box, popular in the late 1800s and early 1900s. It operates under the same principles as those that you might see that are cylindrical. But I am particular impressed with the effort that would go into developing these discs in the pre-computer era, as of course the holes at the outer edge of the disc move faster than the inner ones. It would certainly take a lot of careful calculation to produce one of these. I found this one in the Cray House Museum:
An Arduino Project with Jacob
One day, Jacob and I got going with an Arduino project. He wanted flashing blue lights for his police station , so we disassembled our previous Arduino project, put a few things on the breadboard, I wrote some code, and there we go. Then he noticed an LCD in my Arduino kit. I hadn t ever gotten around to using it yet, and of course he wanted it immediately. So I looked up how to connect it, found an API reference, and dusted off my C skills (that was fun!) to program a scrolling message on it. Here is Jacob showing it off:
What happened in the Reproducible
Builds effort between June 5th and June 11th 2016:
Media coverage
Ed Maste gave a talk at BSDCan 2016 on
reproducible builds
(slides,
video).
GSoC and Outreachy updates
Weekly reports by our participants:
Scarlett Clark
worked on making some packages reproducible, focusing on KDE backend and
utility programs.
Ceridwen
published an initial design for the interface for reprotest, including a
discussion on different types of build variations and the difficulties of
specifying certain types of variations.
Valerie Young improved documentation for
building our tests website, began migrating Debian-specific pages into a new
namespace, and planned future work around its navigation.
Documentation update
- Ximin Luo proposed a modification
to our SOURCE_DATE_EPOCH spec explaining FORCE_SOURCE_DATE.
Some upstream build tools (e.g. TeX, see below) have expressed a desire to
control which cases of embedded timestamps should obey SOURCE_DATE_EPOCH.
They were not convinced by our arguments on why this is a bad idea, so we
agreed on an environment variable FORCE_SOURCE_DATE for them to implement
their desired behaviour - named generically, so that at least we can set it
centrally. For more details, see the text just linked. However, we strongly
urge most build tools not to use this, and instead obey SOURCE_DATE_EPOCHunconditionally in all cases.
Toolchain fixes
TeX Live 2016 released
with SOURCE_DATE_EPOCH support for all engines except LuaTeX and original TeX.
Continued discussion
(alternative archive)
with TeX upstream, about SOURCE_DATE_EPOCH corner cases, eventually
resulting in the FORCE_SOURCE_DATE proposal from above.
gcc-5/5.4.0-4 by Matthias Klose now avoids storing
-fdebug-prefix-map in DW_AT_producer, thanks to original patch by
Daniel Kahn Gillmor.
sphinx/1.4.3-1 by Dmitry Shachnev now drops Debian-specific patches
relating to SOURCE_DATE_EPOCH applied upstream, original patch by Alexis
Bienven e.
asciidoctor/1.5.4-2 by C dric Boutillier now supports
SOURCE_DATE_EPOCH, thanks to original patch by Alexis Bienven e.
dh-python/1.5.4-2 by Piotr O arowski now behaves better in some
cases, thanks to original patch by Chris Lamb.
Andrew Ayer uploaded strip-nondeterminism/0.018-2 fixing #826700, a packaging improvement for Multi-Arch to ease cross-build
situations.
2 days later Andrew released strip-nondeterminism/0.019; now
strip-nondeterminism is able to:
recursively normalize JAR files embedded within JAR files (#823917)
clamp the timestamp, the same way tar >=1.28-2.2 can (for now available only for gzip archives)
disorderfs development
Andrew Ayer released disorderfs/0.4.3, fixing a issue with umask handling (#826891)
tests.reproducible-builds.org
Valerie Young namespaced the Debian-specific pages to /debian/
namespace, with redirects to
for the previous URLs.
Holger Levsen improved the reliability of build jobs: the availability of
both build nodes (for a given build) is now being tested when a build job is
started, to better cope when one of the 25 build nodes go down for some reason.
Ximin Luo improved the index of identified
issues to
include the total popcon scores of each issue, which is now also used for
sorting that page.
Misc.
Steven Chamberlain submitted a patch to FreeBSD's
makefs
to allow reproducible builds of the kfreebsd installer.
Ed Maste committed a patch to FreeBSD's
binutils to
enable determinstic archives by default in GNU ar.
Helmut Grohne
experimentedwith
cross+native reproductions of dash with some success, using
rebootstrap.
This week's edition was written by Ximin Luo, Chris Lamb, Holger Levsen, Mattia
Rizzolo and reviewed by a bunch of Reproducible builds folks on IRC.
Yesterday I was fixing an issue with one of the servers behind kallithea-scm.org: the hook intended to propagage pushes from Our Own Kallithea to Bitbucket stopped working. Until yesterday, that server was using Debian s flavour of System V init and djb s d montools to keep things running. To make the hook asynchronous, I wrote a service to be managed to d montools, so that concurrency issued would be solved by it. However, I didn t implement any timeouts, so when last week wget froze while pulling Weblate s hook, there was nothing to interrupt it, so the hook stopped working since d montools thought it s already running and wouldn t re-trigger it. Killing wget helped, but I decided I need to do something with it to prevent the situation from happening in the future.
I ve been using systemd at work for the last year, so I am now confident I m happier with systemd than with d montools, so I decided to switch the server to systemd. Not surprisingly, I prepared unit files in about 5 minutes without having to look into the manuals again, while with d montools I had to check things every time I needed to change something. The tricky thing was the switch itself. It is a virtual server, presumably running in Xen, and I don t have access to the console, so if I bork break something, I need to summon Bradley Kuhn or someone from Conservancy, who s kindly donated the server to the project. In any case, I decided to attempt to upgrade without a reboot, so that I have more options to roll back my changes in the case things go wrong.
After studying the manpages of both systemd s init and sysvinit s init, I realised I can install systemd as /sbin/init and ask already running System V init to re-exec. However, systemd s init can t talk to System V init, so before installing systemd I made a backup on it. It s also important to stop all running services (except probably ssh) to make sure systemd doesn t start second instances of each. And then: /tmp/init u and we re running systemd! A couple of additional checks, and it s safe to reboot.
Only when I did all that I realised that in the case of systemd not working I d probably not be able to undo my changes if my connection interrupted. So, even though at the end it worked, probably it s not a good idea to perform such manipulations when you don t have an alternative way to connect to the server :)
When you get involved in a new community, such as Debian, find out early
where, if that happens, you can find support, understanding, and help to make
it stop.
Last night I asked a group of friends what do they do if they start feeling
uncomfortable when they are alone in a group of people they don't know. Here
are the replies:
Wait outside the group until I figure the group out.
Find someone to talk for a while until you get comfortable.
If a person is making things uncomfortable for you, let them know, and leave
if nobody cares.
Sit there in silence.
Work around unwelcome people by bearing them for a bit while trying to
integrate with others.
Some people are easy to bribe into friendship, just bring cake.
While you don't know what is going on, you try to replicate what others are
doing.
Spend time trying to get a feeling of what are the consequences of taking
actions.
Purposefully disagree with people in a new environment to figure out if
having a different opinion is accepted.
Once I was new and I was asked to be the person that invites everyone for
lunch, that forced me to talk to everyone, and integrate.
When you are the first one to point something out, you'll probably soon find
out you're not alone.
The reaction on the first time something is exposed, influences how often
similar cases will be reported.
I think a lot of these point are good food for thought about barriers of entry,
and about safety nets that a group has or might want to have.
Allison Lortie has provoked a lot of comment with her blog post on
a new proposal for how GTK is versioned.
Here's some more context from the discussion at the GTK hackfest that
prompted that proposal: there's actually quite a close analogy in
how new Debian versions are developed.
The problem we're trying to address here is the two sides of a trade-off:
Without new development, a library (or an OS) can't go anywhere new
New development sometimes breaks existing applications
Historically, GTK has aimed to keep compatible within a major version,
where major versions are rather far apart (GTK 1 in 1998, GTK 2 in
2002, GTK 3 in 2011, GTK 4 somewhere in the future). Meanwhile, fixing
bugs, improving performance and introducing new features sometimes
results in major changes behind the scenes. In an ideal world, these
behind-the-scenes changes would never break applications; however, the
world isn't ideal. (The Debian analogy here is that as much as we
aspire to having the upgrade from one stable release to the next not break
anything at all, I don't think we've ever achieved that in practice -
we still ask users to read the release notes, even though ideally
that wouldn't be necessary.)
In particular, the perceived cost of doing a proper ABI break (a fully
parallel-installable GTK 4) means there's a strong temptation to make
changes that don't actually remove or change C symbols, but are clearly
an ABI break, in the sense that an application that previously worked
and was considered correct no longer works. A prominent recent example is
the theming changes in GTK 3.20: the ABI in terms of functions available
didn't change, but what happens when you call those functions changed in
an incompatible way. This makes GTK hard to rely on for applications
outside the GNOME release cycle, which is a problem that needs to
be fixed (without stopping development from continuing).
The goal of
the plan we discussed today
is to decouple the latest branch of development, which moves fast and
sometimes breaks API, from the API-stable branches, which only get bug
fixes. This model should look quite familiar to Debian contributors,
because it's a lot like the way we release Debian and Ubuntu.
In Debian, at any given time we have a development branch
(testing/unstable) - currently "stretch", the future Debian 9. We
also have some stable branches, of which the most recent are Debian
8 "jessie" and Debian 7 "wheezy". Different users of Debian have
different trade-offs that lead them to choose one or the other of
these. Users who value stability and want to avoid unexpected changes,
even at a cost in terms of features and fixes for non-critical bugs,
choose to use a stable release, preferably the most recent; they only
need to change what they run on top of Debian for OS API changes (for
instance webapps, local scripts, or the way they interact with the GUI)
approximately every 2 years, or perhaps less often than that with the
Debian-LTS project supporting non-current stable releases. Meanwhile,
users who value the latest versions and are willing to work with a
"moving target" as a result choose to use testing/unstable.
The GTK analogy here is really quite close. In the new versioning model,
library users who value stability over new things would prefer to use
a stable-branch, ideally the latest; library users who want the latest
features, the latest bug-fixes and the latest new bugs would use the
branch that's the current focus of development. In practice we expect
that the latter would be mostly GNOME projects. There's been some
discussion at the hackfest about how often we'd have a new
stable-branch: the fastest rate that's been considered is a stable-branch
every 2 years, similar to Ubuntu LTS and Debian, but there's no consensus
yet on whether they will be that frequent in practice.
How many stable versions of GTK would end up shipped in Debian depends
on how rapidly projects move from "old-stable" to "new-stable" upstream,
how much those projects' Debian maintainers are willing to patch them
to move between branches, and how many versions the release team will
tolerate. Once we reach a steady state, I'd hope that we might have
1 or 2 stable-branched versions active at a time, packaged as separate
parallel-installable source packages (a lot like how we handle Qt).
GTK 2 might well stay around as an additional active version just
from historical inertia. The stable versions are intended to be
fully parallel-installable, just like the situation with GTK 1.2,
GTK 2 and GTK 3 or with the major versions of Qt.
For the "current development" version, I'd anticipate that we'd probably
only ship one source package, and do ABI transitions for one version
active at a time, a lot like how we deal with libgnome-desktop and the
evolution-data-server family of libraries. Those versions would have
parallel-installable runtime libraries but non-parallel-installable
development files, again similar to libgnome-desktop.
At the risk of stretching the Debian/Ubuntu analogy too far, the
intermediate "current development" GTK releases that would accompany
a GNOME release are like Ubuntu's non-LTS suites: they're more up to
date than the fully stable releases (Ubuntu LTS, which has a release
schedule similar to Debian stable), but less stable and not supported
for as long.
Hopefully this plan can meet both of its goals: minimize breakage for
applications, while not holding back the development of new APIs.
Author: ceridwen
The first, very-alpha release of reprotest is now out at
PyPi. It should hit
Debian experimental later this week. While it only builds on an
existing system (as I'm still working on support for virtualization),
it can now check its own reproducibility, which it does in its own
tests, both using setuptools and debuild. Unfortunately, setuptools
seems to generate file-order-dependent binaries, meaning python
setup.py bdist creates unreproducible binaries. With debuild,
reprotest probably would be reproducible with the modified
packages
from the Reproducible Builds project, though I haven't tested that
yet. It tests 'captures_environment', 'fileordering' (renamed from
'filesystem'), 'home', 'kernel', 'locales', 'path', 'time',
'timezone', and 'umask'. The other variations require superuser
privileges and modifications that would be unsafe to make to a running
system, so they will only be enabled in the containers.
The next major part of the project is integrating autopkgtests's
container management system into reprotest. For the curious,
autopkgtest is composed of a main program, adt-run, which then calls
other command-line programs, adt-virt-chroot, adt-virt-lxd,
adt-virt-schroot, adt-virt-null, adt-virt-schroot, and
adt-virt-qemu, that communicate with the containers. (The
autopkgtest maintainer has since renamed the programs, but the
underlying structure remains the same.) I think this is a bit of an
odd design but it works well for my purposes since the container
programs already have existing CLIs that reprotest can use.
I begin this week by finish my QT tutorial. With that new knowledge, I was able to implemet my method launchSmartInfo(int) on LRC.
After that, I needed to implement the gnome client too. I followed the tutorial on QT so I thought I can just learn GTK+ by reading the code. Finally, I just lost a lot of time by doing that and I didn't learn a lot. In response to that problem I finally use an GTK+ tutorial.
I began to try to show n transparente window with some text in front of the call view. To do that, I want to use clutter library
------------
Conclusion: as you can see, I lost a lot of time in this GUI learning. Now it's done, I can move forward! :)
I begin this week by finish my QT tutorial. With that new knowledge, I was able to implemet my method launchSmartInfo(int) on LRC.
After that, I needed to implement the gnome client too. I followed the tutorial on QT so I thought I can just learn GTK+ by reading the code. Finally, I just lost a lot of time by doing that and I didn't learn a lot. In response to that problem I finally use an GTK+ tutorial.
I began to try to show an transparente window with some text in front of the call view. To do that, I want to use clutter library
------------
Conclusion: as you can see, I lost a lot of time in this GUI learning. Now it's done, I can move forward! :)
This has been approved and I will be pushing it as soon as I patch the qt5 frameworks version.
kxmlgui:
WIP this has been a steep learning curve, according to the notes it was an easy embedded kernel version, that was not the case! After grueling hours of
trying to sort out randomness in debug output I finally narrowed it down to cases where QStringLiteral was used and there were non letter characters eg. ( <")
These were causing debug symbols to generate with ( lambda() ) which caused unreproducible symbol/debug files. It is now a case of fixing all of these in the code to use QString::fromUtf8 seems to fix this. I am working on a mega patch for upstream and it should be ready early in the week.
This last week I spent a large portion making my through a mega patch for kxmlgui, when it was suggested to me to write a small qt app
to test QStringLiteral isolated and sure enough two build were byte for byte identical. So this means that QStringLiteral may not be the issue at all. With some
more assistance I am to expand my test app with several QStringLiterals of varying lengths, we have suspicion it is a padding issue, which complicates things.
KDE:
On the KDE front, I have arrived safe and sound in Randa and aside from some major jetlag, reproducible builds, I have been quite busy with the KDE CI. I am reworking
my DSL to use friendly yaml files to generate jobs for all platforms ( linux, android, osx, windows, snappy, flatpak ) and can easily be extended later.
Major workpoints so far for Randa:
I have delegated the windows backend to Hannah
Andreas has provided a docker build for Android, and upon initial testing it will work great.
I have recruited several nice folks to assist me with my snappy efforts.