wadclicommand-line interface respecting preferences and a new preferences UI dialog (adapted from Quake Injector). There are two new example maps: A Labyrinth demonstration contributed by "Yoruk", and a Heretic map Bird Cage by yours truly. These are both now amongst the largest examples in the collection, although
laby.wlwas generated by a higher-level program. For more information see the release notes and the reference, or check out the new gallery of examples or skip straight to downloads. I have no plans to work on WadC further (but never say never, I suppose.)
Since for a student it is better to have everything for experiments, I install the full version, not only the
apt-get install openjdk-8-jdk
-headlessversion. Given my familiarity with Debian/Ubuntu, I didn't have to think about the way of installing it, of course. But as this is a tutorial meant to be as general as I can, I tried also to include instructions on how to install Java on other distributions. The first two that came to my mind were openSUSE and Fedora. Both use the RPM package format for their "native" packages (in the same sense that Debian uses DEB packages for "native" packages). But they use different higher-level tools to install such packages: Fedora uses a tool called
dnf, while openSUSE uses
zypper. To try these distributions, I got their netinstall ISOs and used
kvmto install on a virtual machine. I used the following to install/run the virtual machines (the example below, is, of course, for openSUSE):
The names of the packages also change from one distribution to another. On Fedora, I had to use:
qemu-system-x86_64 -enable-kvm -m 4096 -smp 2 -net nic,model=e1000 -net user -drive index=0,media=disk,cache=unsafe,file=suse.qcow2 -cdrom openSUSE-Leap-42.3-NET-x86_64.iso
On openSUSE, I had to use:
dnf install java-1.8.0-openjdk-devel
Note that one distribution uses dots in the names of the packages while the other uses underscores. One interesting thing that I noticed with
zypper install java-1_8_0-openjdk-devel
dnfwas that, when I used it, it automatically refreshed the package lists from the network, something which I forgot, and it was a pleasant surprise. I don't know about
zypper, but I guess that it probably had fresh indices when the installation finished. Both installations were effortless after I knew the names of the packages to install. Oh, BTW, in my 5 minute exploration with these distributions, I noticed that if you don't want the JDK, but only the JRE, then you omit the
-develsuffix. It makes sense when you think about it, for consistency with other packages, but Debian's conventions also make sense (JRE with
-jresuffix, JDK with
-jdksuffix). I failed miserably to use Fedora's prebaked, vanilla cloud image, as I couldn't login on this image and I decided to just install the whole OS on a fresh virtual machine. I don't have instructions on how to install on Gentoo nor on Arch, though. I now see how hard it is to cover instructions/provide software for as many distributions as you wish, given the multitude of package managers, conventions etc.
f :: a bbetween two types
b(the function arrow in Isabelle is
), then by definition of what it means to be a function in HOL whenever I have a value
x :: a, then the expression
x) is a value of type
b. Therefore, and without exception, every Isabelle function is total. In particular, it cannot be that
f xdoes not exist for some
x :: a. This is a first difference from Haskell, which does have partial functions like
Here, neither the expression
spin :: Maybe Integer -> Bool spin (Just n) = spin (Just (n+1))
spin Nothingnor the expression
spin (Just 42)produce a value of type
Bool: The former raises an exception ( incomplete pattern match ), the latter does not terminate. Confusingly, though, both expressions have type
Bool. Because every function is total, this confusion cannot arise in Isabelle: If an expression
t, then it is a value of type
t. This trait is shared with other total systems, including Coq. Did you notice the emphasis I put on the word is here, and how I deliberately did not write evaluates to or returns ? This is because of another big source for confusion:
a bin functional programming is an algorithm that, given a value of type
a, calculates (returns, evaluates to) a value of type
a bin math (or Isabelle) associates with each value of type
aa value of type
This assigns a real number to every sequence, but it does not compute it in any useful sense. From this it follows that
definition foo :: "(nat real) real" where "foo seq = (if convergent seq then lim seq else 0)"
To a functional programmer, this reads
fun plus :: "nat nat nat" where "plus 0 m = m" "plus (Suc n) m = Suc (plus n m)"
which is clearly a description of a computation. But to Isabelle, the above reads
plusis a function that analyses its first argument. If that is
0, then it returns the second argument. Otherwise, it calls itself with the predecessor of the first argument and increases the result by one.
And in fact, it is not so much Isabelle that reads it this way, but rather the
plusis a binary function on natural numbers, and it satisfies the following two equations:
funcommand, which is external to the Isabelle logic. The
funcommand analyses the given equations, constructs a non-recursive definition of
plusunder the hood, passes that to Isabelle and then proves that the given equations hold for
plus. One interesting consequence of this is that different specifications can lead to the same functions. In fact, if we would define
plus'by recursing on the second argument, we d obtain the the same function (i.e.
plus = plus'is a theorem, and there would be no way of telling the two apart).
funcommand can only construct
plusin a way that the equations hold if it passes a termination check very much like
Fixpointin Coq. But while the termination check of
Fixpointin Coq is a deep part of the basic logic, in Isabelle it is simply something that this particular command requires for its internal machinery to go through. At no point does a termination proof of the function exist as a theorem inside the logic. And other commands may have other means of defining a function that do not even require such a termination argument! For example, a function specification that is tail-recursive can be turned in to a function, even without a termination proof: The following definition describes a higher-order function that iterates its first argument
fon the second argument
xuntil it finds a fixpoint. It is completely polymorphic (the single quote in
'aindicates that this is a type variable):
We can work with this definition just fine. For example, if we instantiate
partial_function (tailrec) fixpoint :: "('a 'a) 'a 'a" where "fixpoint f x = (if f x = x then x else fixpoint f (f x))"
( x. x-1), we can prove that it will always return 0:
Similarly, if we have a function that works within the option monad (i.e. Maybe in Haskell), its specification can always be turned into a function without an explicit termination proof here one that calculates the Collatz sequence:
lemma "fixpoint ( n . n - 1) (n::nat) = 0" by (induction n) (auto simp add: fixpoint.simps)
Note that lists in Isabelle are finite (like in Coq, unlike in Haskell), so this function returns a list only if the collatz sequence eventually reaches 1. I expect these definitions to make a Coq user very uneasy. How can
partial_function (option) collatz :: "nat nat list option" where "collatz n = (if n = 1 then Some [n] else if even n then do ns <- collatz (n div 2); Some (n # ns) else do ns <- collatz (3 * n + 1); Some (n # ns) )"
fixpointbe a total function? What is
fixpoint ( n. n+1)? What if we run
collatz nfor a
nwhere the Collatz sequence does not reach 1?3 We will come back to that question after a little detour
The naming of this term alone has caused a great deal of confusion for Isabelle beginners, or in communication with users of different systems, so I implore you to not read too much into the name. In fact, you will have a better time if you think of it as
undefined :: 'a
arbitraryor, even better,
undefinedcan be instantiated at any type, we can instantiate it for example at
bool, and we can observe an important fact:
undefinedis not an extra value besides the usual ones . It is simply some value of that type, which is demonstrated in the following lemma:
In fact, if the type has only one value (such as the unit type), then we know the value of
lemma "undefined = True undefined = False" by auto
It is very handy to be able to produce an expression of any type, as we will see as follows
lemma "undefined = ()" by auto
This definition is accepted by
fun fromSome :: "'a option 'a" where "fromSome (Some x) = x"
fun(albeit with a warning), and the generated function
fromSomebehaves exactly as specified: when applied to
Some x, it is
x. The term
fromSome Noneis also a value of type
'a, we just do not know which one it is, as the specification does not address that. So
fromSome Nonebehaves just like
undefinedabove, i.e. we can prove
Here is a small exercise for you: Can you come up with an explanation for the following lemma:
lemma "fromSome None = False fromSome None = True" by auto
Overall, this behavior makes sense if we remember that function definitions in Isabelle are not really definitions, but rather specifications. And a partial function definition is simply a underspecification. The resulting function is simply any function hat fulfills the specification, and the two lemmas above underline that observation.
fun constOrId :: "bool bool" where "constOrId True = True" lemma "constOrId = ( _.True) constOrId = ( x. x)" by (metis (full_types) constOrId.simps)
fixpointabove. Clearly, the function seen as a functional program is not total: When passed the argument
( n. n + 1)or
( b. b)it will loop forever trying to find a fixed point. But Isabelle functions are not functional programs, and the definitions are just specifications. What does the specification say about the case when
fhas no fixed-point? It states that the equation
fixpoint f x = fixpoint f (f x)holds. And this equation has a solution, for example
fixpoint f _ = undefined. Or more concretely: The specification of the
fixpointfunction states that
fixpoint ( b. b) True = fixpoint ( b. b) Falsehas to hold, but it does not specify which particular value (
False) it should denote any is fine.
fand get a function out of that? But rest assured: That is not the case. For example, no Isabelle command allows you define a function
bogus :: () natwith the equation
bogus () = Suc (bogus ()), because this equation does not have a solution. We can actually prove that such a function cannot exist:
lemma no_bogus: " bogus. bogus () = Suc (bogus ())" by simp
not_bogus () = not_bogus ()is just fine )
undefinedis not a separate, recognizable value, but rather simply an unknown one, there is no way of stating that A function result is not specified . Here is an example that demonstrates this: Two partial functions (one with not all cases specified, the other one with a self-referential specification) are indistinguishable from the total variant:
If you really do want to reason about partiality of functional programs in Isabelle, you should consider implementing them not as plain HOL functions, but rather use HOLCF, where you can give equational specifications of functional programs and obtain continuous functions between domains. In that setting,
fun partial1 :: "bool unit" where "partial1 True = ()" partial_function (tailrec) partial2 :: "bool unit" where "partial2 b = partial2 b" fun total :: "bool unit" where "total True = ()" "total False = ()" lemma "partial1 = total partial2 = total" by auto
partial2 = total. We have done that to verify some of HLint s equations.
fixpoint, which is not possible in Coq. There is currently ongoing work about verified code generation, where the code equations are reflected into a deep embedding of HOL in Isabelle that would allow explicit termination proofs.
undefinedin Coq This section is speculative, and an invitation for discussion. Coq already distinguishes between types used in programs (
Set) and types used in proofs
Prop. Could Coq ensure that every
t : Setis non-empty? I imagine this would require additional checks in the
Inductivecommand, similar to the checks that the Isabelle command
datatypehas to perform4, and it would disallow
Empty_set. If so, then it would be sound to add the following axiom
wouldn't it? This axiom does not have any computational meaning, but that seems to be ok for optional Coq axioms, like classical reasoning or function extensionality. With this in place, how much of what I describe above about function definitions in Isabelle could now be done soundly in Coq. Certainly pattern matches would not have to be complete and could sport an implicit case
Axiom undefined : forall (a : Set), a.
_ undefined. Would it help with non-obviously terminating functions? Would it allow a Coq command
Tailrecursivethat accepts any tailrecursive function without a termination check?
fun, the constructions by
datatypeare not part of the logic, but create a type definition from more primitive notions that is isomorphic to the specified data type.
autopkgtest [21:47:38]: test nm: [-----------------------
ethernet: auto-connection, IPv4 ... FAIL ----- NetworkManager.log -----
NetworkManager: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.25' not found (required by NetworkManager)
$ grep 2.25 ./sysdeps/unix/sysv/linux/x86_64/64/libc.abilist
GLIBC_2.25 GLIBC_2.25 A
GLIBC_2.25 __explicit_bzero_chk F
GLIBC_2.25 explicit_bzero F
GLIBC_2.25 getentropy F
GLIBC_2.25 getrandom F
GLIBC_2.25 strfromd F
GLIBC_2.25 strfromf F
GLIBC_2.25 strfroml F
$ grep explicit_bzero -r configure.ac src/
src/systemd/src/basic/string-util.h:void explicit_bzero(void *p, size_t l);
src/systemd/src/basic/string-util.c:void explicit_bzero(void *p, size_t l)
src/systemd/src/basic/string-util.c: explicit_bzero(x, strlen(x));
<provides/>-tag. Naming fonts and making them identifiable is a whole other issue, I used a document from Adobe on font naming issues as a rough guideline while working on this. How to write a good metainfo file for a font is best shown with an example. Lato is a well-looking font family that we want displayed in a software center. So, we write a metainfo file for it an place it in
/usr/share/metainfo/com.latofonts.Lato.metainfo.xmlfor the AppStream metadata generator to pick up:
<?xml version="1.0" encoding="UTF-8"?> <component type="font"> <id>com.latofonts.Lato</id> <metadata_license>FSFAP</metadata_license> <project_license>OFL-1.1</project_license> <name>Lato</name> <summary>A sanserif type face fam ily</summary> <description> <p> Lato is a sanserif type face fam ily designed in the Sum mer 2010 by Warsaw-based designer ukasz Dziedzic ( Lato means Sum mer in Pol ish). In Decem ber 2010 the Lato fam ily was pub lished under the open-source Open Font License by his foundry tyPoland, with sup port from Google. </p> </description> <url type="homepage">http://www.latofonts.com/</url> <provides> <font>Lato Regular</font> <font>Lato Black Italic</font> <font>Lato Black</font> <font>Lato Bold Italic</font> <font>Lato Bold</font> <font>Lato Hairline Italic</font> ... </provides> </component>When the file is processed, we know that we need to look for fonts in the package it is contained in. So, the appstream-generator will load all the fonts in the package and render example texts for them as an image, so we can show users a preview of the font. It will also use heuristics to render an icon for the respective font component using its regular typeface. Of course that is not ideal what if there are multiple font faces in a package? What if the heuristics fail to detect the right font face to display? This behavior can be influenced by adding
<font/>tags to a
<provides/>tag in the metainfo file. The font-provides tags should contain the fullnames of the font faces you want to associate with this font component. If the font file does not define a fullname, the family and style are used instead. That way, someone writing the metainfo file can control which fonts belong to the described component. The metadata generator will also pick the first mentioned font name in the
<provides/>list as the one to render the example icon for. It will also sort the example text images in the same order as the fonts are listed in the provides-tag. The example lines of text are written in a language matching the font using Pango. But what about symbolic fonts? Or fonts where any heuristic fails? At the moment, we see ugly tofu characters or boxes instead of an actual, useful representation of the font. This brings me to an inofficial extension to font metainfo files, that, as far as I know, only appstream-generator supports at the moment. I am not happy enough with this solution to add it to the real specification, but it serves as a good method to fix up the edge cases where we can not render good example images for fonts. AppStream-Generator supports the FontIconText and FontSampleText custom AppStream properties to allow metainfo file authors to override the default texts and autodetected values. FontIconText will override the characters used to render the icon, while FontSampleText can be a line of text used to render the example images. This is especially useful for symbolic fonts, where the heuristics usually fail and we do not know which glyphs would be representative for a font. For example, a font with mathematical symbols might want to add the following to its metainfo file:
<custom> <value key="FontIconText"> </value> <value key="FontSampleText"> ... </value> </custom>Any unicode glyphs are allowed, but asgen will but some length restrictions on the texts. So, In summary:
NEWSfile entry below lists all changes.
Courtesy of CRANberries, there is also a diffstat report for the most recent release. More information is on the RcppZiggurat page.
Changes in version 0.1.4 (2017-07-27)
- The vignette now uses the pinp package in two-column mode.
- Dynamic symbol registration is now enabled.
setup_requiresto pull in pbr, and so we ve been stuck on old versions for some time; this is part of why Launchpad doesn t yet support newer SSH key types, for instance. This situation obviously isn t sustainable. To deal with this, I ve been working for some time on switching to virtualenv and pip. This is harder than you might think: Launchpad is a long-lived and complicated project, and it had quite a number of explicit and implicit dependencies on Buildout s configuration and behaviour. Upgrading our infrastructure from Ubuntu 12.04 to 16.04 has helped a lot (12.04 s baseline virtualenv and pip have some deficiencies that would have required a more complicated bootstrapping procedure). I ve dealt with most of these: for example, I had to reorganise a lot of our helper scripts (1, 2, 3), but there are still a few more things to go. One remaining problem was that our Buildout configuration relied on building several different environments with different Python paths for various things. While this would technically be possible by way of building multiple virtualenvs, this would inflate our build time even further (we re already going to have to cope with some slowdown as a result of using virtualenv, because the build system now has to do a lot more than constructing a glorified link farm to a bunch of cached eggs), and it seems like unnecessary complexity. The obvious thing to do seemed to be to collapse these into a single environment, since there was no obvious reason why it should actually matter if txpkgupload and txlongpoll were carefully kept off the path when running most of Launchpad: so I did that. Then our build system got very sad. Hmm, I thought. To keep our test times somewhat manageable, we run them in parallel across 20 containers, and we randomise the order in which they run to try to shake out test isolation bugs. It s not completely unknown for there to be some oddities resulting from that. So I ran it again. Nope, but slightly differently sad this time. Furthermore, I couldn t reproduce these failures locally no matter how hard I tried. Oh dear. This was obviously not going to be a good day. In fact I spent a while on various different guesswork-based approaches. I found bug 571334 in Ampoule, an AMP-based process pool implementation that we use for some job runners, and proposed a fix for that, but cherry-picking that fix into Launchpad didn t help matters. I tried backing out subsets of my changes and determined that if both
txpkguploadwere absent from the Python module path in the context of the tests in question then everything was fine. I tried running
stracelocally and staring at the output for some time in the hope of enlightenment: that reminded me that the two packages in question install modules under
twisted.plugins, which did at least establish a reason they might affect the environment that was more plausible than magic, but nothing much more specific than that. On Friday I was fiddling about with this again and trying to insert some more debugging when I noticed some interesting behaviour around plugin caching. If I caused the
txpkguploadplugin to raise an exception when loaded, the Twisted plugin system would remove its
dropin.cache(because it was stale) and not create a new one (because there was now no content to put in it). After that, running the relevant tests would fail as I d seen in our buildbot. Aha! This meant that I could also reproduce it by doing an even cleaner build than I d previously tried to do, by removing the cached
txlongpolleggs and allowing the build system to recreate them. When they were recreated, they didn t contain
dropin.cache, instead allowing that to be created on first use. Based on this clue I was able to get to the answer relatively quickly. Ampoule has a specialised bootstrapping sequence for its worker processes that starts by doing this:
from twisted.application import reactors reactors.installReactor(reactor)
twisted.plugin.getPlugins, so the very start of this bootstrapping sequence is going to involve loading all plugins found on the module path (I assume it s possible to write a plugin that adds an alternative reactor implementation). If
dropin.cacheis up to date, then it will just get the information it needs from that; but if it isn t, it will go ahead and import the plugin. If the plugin happens (as Twisted code often does) to run
from twisted.internet import reactorat some point while being imported, then that will install the platform s default reactor, and then
ReactorAlreadyInstalledError. Since Ampoule turns this into an info-level log message for some reason, and the tests in question only passed through error-level messages or higher, this meant that all we could see was that a worker process had exited non-zero but not why. The Twisted documentation recommends generating the plugin cache at build time for other reasons, but we weren t doing that. Fixing that makes everything work again. There are still a few more things needed to get us onto pip, but we re now pretty close. After that we can finally start bringing our dependencies up to date.
$ apt install devscripts [ ] $ reproducible-check [ ] W: subversion (1.9.7-2) is unreproducible (libsvn-perl, libsvn1, subversion) <https://tests.reproducible-builds.org/debian/subversion> W: taglib (1.11.1+dfsg.1-0.1) is unreproducible (libtag1v5, libtag1v5-vanilla) <https://tests.reproducible-builds.org/debian/taglib> W: tcltk-defaults (8.6.0+9) is unreproducible (tcl, tk) <https://tests.reproducible-builds.org/debian/tcltk-defaults> W: tk8.6 (8.6.7-1) is unreproducible (libtk8.6, tk8.6) <https://tests.reproducible-builds.org/debian/tk8.6> W: valgrind (1:3.13.0-1) is unreproducible <https://tests.reproducible-builds.org/debian/valgrind> W: wavpack (5.1.0-2) is unreproducible (libwavpack1) <https://tests.reproducible-builds.org/debian/wavpack> W: x265 (2.5-2) is unreproducible (libx265-130) <https://tests.reproducible-builds.org/debian/x265> W: xen (4.8.1-1+deb9u1) is unreproducible (libxen-4.8, libxenstore3.0) <https://tests.reproducible-builds.org/debian/xen> W: xmlstarlet (1.6.1-2) is unreproducible <https://tests.reproducible-builds.org/debian/xmlstarlet> W: xorg-server (2:1.19.3-2) is unreproducible (xserver-xephyr, xserver-xorg-core) <https://tests.reproducible-builds.org/debian/xorg-server> 282/4494 (6.28%) of installed binary packages are unreproducible.
$ reproducible-check --raw dd-list --stdin Alec Leamas <firstname.lastname@example.org> lirc (U) Alessandro Ghedini <email@example.com> valgrind Alessio Treglia <firstname.lastname@example.org> fluidsynth (U) libsoxr (U) [ ]
[$] dpkg -L youtube-dl grep youtube.pyHopefully somebody does the needful. Btw, I find f-droid extremely useful and especially osmand but sadly both of them are not shared or talked about by people The reason I shared about Developer Options in Android is that few days back I noticed that the phone wonks off and has unpredictable behaviour such as not letting me browse the web, do additions or deletions using the google play store and alike . Things came to a head when few days back I saw a fibre-optic splicing operation being carried by some workers near my home by the state operator which elated me and wanted to shoot the video for it but the battery died/there was no power even though I hadn t used it much. I have deliberately shared the hindi version which tells how that knowledge is now coming to the masses. I had seen fibre-optic splicing more than a decade and a half back at some net conference where it was going to be in your neighbourhood soonish, hopefully it will happen soon I had my suspicions for quite sometime all the issues with the phone were not due to proper charging. During course of my investigation, found out that in Developer Options there is an option called USB Configuration and changing that from the default MTP (Media Transfer Protocol) (which is basically used to put or take movies, music or any file from the phone to the computer or vice-versa improved much better behaviour on my android phone. But this caused an unexpected side-effect, I got pretty aggressive polling of the phone by the computer even after saying I do not want to share the phone details with the computer. This I filed as 874216 . The phone and I am guessing most Samsung phones today come with an adaptor with a USB male socket which goes in to the phone s usb port. There is the classical port for electricity but like most people heavily rely on usb charging even for deep fully powered down phone for full charging. One interesting project which I saw which came in Debian some days back is dummydroid. I did file a bug about it . I do hope that either the maintainer gives some more documentation. I am sure many people might use and add to the resource if the documentation was/is there. I did take a look at the package and the profile seems to be like an xml pair kind of database. Having more profiles shouldn t be hard if we knew what info. needs to be added and how do we find that info. Lastly, I am slowly transferring all the above knowledge to my mum as well, although in small doses. She, just like me has and had problems coming from resistive touchscreen to capacitive touchscreen. You can call me wrong but resistive touchscreen seemed to be superior and not as error-prone or liable to commit mistakes as is possible in capacitive touchscreens. There may be a setting to higher/lower the threshold for touching which I have not been able to find as of yet. Hope somebody finds something useful in there. I do hope that Debian does become a replacement to be used on such mobiles but then they would have to duplicate/also have some sort of mainstream content with editors to help people find stuff, something that Debian is not so good at currently. Also I m not sure Synaptic is good fit as a mobile store.
sudo apt install \ git \ rake \ libvirt-daemon-system \ dnsmasq-base \ ebtables \ qemu-system-x86 \ qemu-utils \ vagrant \ vagrant-libvirt \ vmdebootstrap && \ sudo systemctl restart libvirtd
for group in kvm libvirt libvirt-qemu ; do sudo adduser "$(whoami)" "$group" done
Send us feedback! No matter how your build attempt turned out we are interested in your feedback. Gather system information To gather the information we need about your system, run the following commands in the terminal where you've run
git clone https://git-tails.immerda.ch/tails && \ cd tails && \ git checkout 3.2-alpha2 && \ git submodule update --init && \ rake build
Then check that the generated file doesn't contain any sensitive information you do not want to leak:
sudo apt install apt-show-versions && \ ( for f in /etc/issue /proc/cpuinfo do echo "--- File: $ f ---" cat "$ f " echo done for c in free locale env 'uname -a' '/usr/sbin/libvirtd --version' \ 'qemu-system-x86_64 --version' 'vagrant --version' do echo "--- Command: $ c ---" eval "$ c " echo done echo '--- APT package versions ---' apt-show-versions qemu:amd64 linux-image-amd64:amd64 vagrant \ libvirt0:amd64 ) bzip2 > system-info.txt.bz2
Next, please follow the instructions below that match your situation! If the build failed Sorry about that. Please help us fix it by opening a ticket:
system-info.txt.bz2(this will publish that file).
Compare your checksum with ours:
If the checksums match: success, congrats for reproducing Tails 3.2~alpha2! Please send an email to email@example.com (public) or firstname.lastname@example.org (private) with the subject "Reproduction of Tails 3.2~alpha2 successful" and
system-info.txt.bz2attached. Thanks in advance! Then you can stop reading here. Else, if the checksums differ: too bad, but really it's good news as the whole point of the exercise is precisely to identify such problems :) Now you are in a great position to help improve the reproducibility of Tails ISO images by following these instructions:
diffoscopeversion 83 or higher and all the packages it recommends. For example, if you're using Debian Stretch:
sudo apt remove diffoscope && \ echo 'deb http://ftp.debian.org/debian stretch-backports main' \ sudo tee /etc/apt/sources.list.d/stretch-backports.list && \ sudo apt update && \ sudo apt -o APT::Install-Recommends="true" \ install diffoscope/stretch-backports
diffoscope \ --text diffoscope.txt \ --html diffoscope.html \ --max-report-size 262144000 \ --max-diff-block-lines 10000 \ --max-diff-input-lines 10000000 \ path/to/official/tails-amd64-3.2~alpha2.iso \ path/to/your/own/tails-amd64-3.2~alpha2.iso bzip2 diffoscope. txt,html
diffoscope.html.bz2, except if they are larger than 100 KiB, in which case better upload the file somewhere (e.g. share.riseup.net and share the link in your email.
Documentation/security/tree had been converted yet. I took the opportunity to take a few passes at formatting the existing documentation and, at Jon Corbet s recommendation, split it up between end-user documentation (which is mainly how to use LSMs) and developer documentation (which is mainly how to use various internal APIs). A bunch of these docs need some updating, so maybe with the improved visibility, they ll get some extra attention. CONFIG_REFCOUNT_FULL
refcount_tAPI in v4.11, Elena Reshetova (with Hans Liljestrand and David Windsor) has been systematically replacing
atomic_treference counters with
refcount_t. As of v4.13, there are now close to 125 conversions with many more to come. However, there were concerns over the performance characteristics of the
refcount_timplementation from the maintainers of the net, mm, and block subsystems. In order to assuage these concerns and help the conversion progress continue, I added an unchecked
refcount_timplementation (identical to the earlier
atomic_timplementation) as the default, with the fully checked implementation now available under
CONFIG_REFCOUNT_FULL. The plan is that for v4.14 and beyond, the kernel can grow per-architecture implementations of
refcount_tthat have performance characteristics on par with
atomic_t(as done in grsecurity s PAX_REFCOUNT). CONFIG_FORTIFY_SOURCE
FORTIFY_SOURCEcompile-time and run-time protection for finding overflows in the common string (e.g.
strcmp) and memory (e.g.
memcmp) functions. The idea is that since the compiler already knows the size of many of the buffer arguments used by these functions, it can already build in checks for buffer overflows. When all the sizes are known at compile time, this can actually allow the compiler to fail the build instead of continuing with a proven overflow. When only some of the sizes are known (e.g. destination size is known at compile-time, but source size is only known at run-time) run-time checks are added to catch any cases where an overflow might happen. Adding this found several places where minor leaks were happening, and Daniel and I chased down fixes for them. One interesting note about this protection is that is only examines the size of the whole object for its size (via
__builtin_object_size(..., 0)). If you have a string within a structure,
CONFIG_FORTIFY_SOURCEas currently implemented will make sure only that you can t copy beyond the structure (but therefore, you can still overflow the string within the structure). The next step in enhancing this protection is to switch from 0 (above) to 1, which will use the closest surrounding subobject (e.g. the string). However, there are a lot of cases where the kernel intentionally copies across multiple structure fields, which means more fixes before this higher level can be enabled. NULL-prefixed stack canary
strcpy), since they will either stop an overflowing read at the NULL byte, or be unable to write a NULL byte, thereby always triggering the canary check. This does reduce the entropy from 64 bits to 56 bits for overflow cases where NULL bytes can be written (e.g.
memcpy), but the trade-off is worth it. (Besdies, x86_64 s canary was 32-bits until recently.) IPC refactoring
__randomize_layout). v4.14 will also have the automatic mode enabled, which randomizes all structures that contain only function pointers. A large number of fixes to support randstruct have been landing from v4.10 through v4.13, most of which were already identified and fixed by grsecurity, but many were novel, either in newly added drivers, as whitelisted cross-structure casts, refactorings (like IPC noted above), or in a corner case on ARM found during upstream testing. lower ELF_ET_DYN_BASE
ELF_ET_DYN_BASE(the lowest possible random position of a PIE executable in memory) was already so high in the memory layout (specifically, 2/3rds of the way through the address space). Fixing this required teaching the ELF loader how to load interpreters as shared objects in the mmap region instead of as a PIE executable (to avoid potentially colliding with the binary it was loading). As a result, the PIE default could be moved down to ET_EXEC (0x400000) on 32-bit, entirely avoiding the subset of Stack Clash attacks. 64-bit could be moved to just above the 32-bit address space (0x100000000), leaving the entire 32-bit region open for VMs to do 32-bit addressing, but late in the cycle it was discovered that Address Sanitizer couldn t handle it moving. With most of the Stack Clash risk only applicable to 32-bit, fixing 64-bit has been deferred until there is a way to teach Address Sanitizer how to load itself as a shared object instead of as a PIE binary. early device randomness
2017, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
In Canada "consent means the voluntary agreement of the complainant to engage in sexual activity" without abuse or exploitation of "trust, power or authority", coercion or threats. Consent can also be revoked at any moment. There are 3 pillars often included in the description of sexual consent, or "the way we let others know what we're up for, be it a good-night kiss or the moments leading up to sex." They are:Saying "I've decided I won't do laundry anymore" when the other partner is tired, or busy doing things. Is different than saying "I've decided I won't do laundry anymore" when the other partner has a chance to say "why? tell me more" and take part in negotiation. Resources: Thomas-Kilmann Conflict Mode Instrument: source png. Motivations Quick poll:
- Knowing exactly what and how much I'm agreeing to
- Expressing my intent to participate
- Deciding freely and voluntarily to participate
"Spoons" are a visual representation used as a unit of measure used to quantify how much energy a person has throughout a given day. Each activity requires a given number of spoons, which will only be replaced as the person "recharges" through rest. A person who runs out of spoons has no choice but to rest until their spoons are replenished.
My software ate(inspired by a 1934 poem by William Carlos Williams) Don't be afraid to fail Don't be afraid to fail or drop the ball. I think that anything that has a label attached of "if you don't do it, nobody will", shouldn't fall on anybody's shoulders and should be shared no matter what. Shared or dropped. Share the responsibility for a healthy relationship Don't expect that the more experienced mates will take care of everything. In a project with active people counted by the thousand, it's unlikely that harassment isn't happening. Is anyone writing anti-harassment? Do we have stats? Is having an email address and a CoC giving us a false sense of security?
that where in
your home directory and which
you were probably
for work Forgive me
it was so quick to write
and it worked so well for me
When you get involved in a new community, such as Debian, find out early where, if that happens, you can find support, understanding, and help to make it stop. If you cannot find any, or if the only thing you can find is people who say "it never happens here", consider whether you really want to be in that community.(from http://www.enricozini.org/blog/2016/debian/you-ll-thank-me-later/)
There are some nice people in the world. I mean nice people, the sort I couldn t describe myself as. People who are friends with everyone, who are somehow never involved in any argument, who seem content to spend their time drawing pictures of bumblebees on flowers that make everyone happy. Those people are great to have around. You want to hold onto them as much as you can. But people only have so much tolerance for jerkiness, and really nice people often have less tolerance than the rest of us. The trouble with not ejecting a jerk whether their shenanigans are deliberate or incidental is that you allow the average jerkiness of the community to rise slightly. The higher it goes, the more likely it is that those really nice people will come around less often, or stop coming around at all. That, in turn, makes the average jerkiness rise even more, which teaches the original jerk that their behavior is acceptable and makes your community more appealing to other jerks. Meanwhile, more people at the nice end of the scale are drifting away.(from https://eev.ee/blog/2016/07/22/on-a-technicality/) Give people freedom If someone tries something in Debian, try to acknowledge and accept their work. You can give feedback on what they are doing, and try not to stand in their way, unless what they are doing is actually hurting you. In that case, try to collaborate, so that you all can get what you need. It's ok if you don't like everything that they are doing. I personally don't care if people tell me I'm good when I do something, I perceive it a bit like "good boy" or "good dog". I rather prefer if people show an interest, say "that looks useful" or "how does it work?" or "what do you need to deploy this?" Acknowledge that I've done something. I don't care if it's especially liked, give me the freedom to keep doing it. Don't give me rewards, give me space and dignity. Rather than feeding my ego, feed by freedom, and feed my possibility to create.
INto the host or
OUTto the device. Endpoints are not bidirectional, but the in and out endpoints do overlap in numbering. There is a special pair of endpoints,
OUT0which, between them, form what I will call the device control endpoints. The device control endpoints are important since every USB device MUST implement them, and there are a number of well defined messages which pass over them to control the USB device. In theory a bare minimum USB device would implement only the device control endpoints.
|Field Name||Byte start||Byte length||Encoding||Meaning|
|bLength||0||1||Number||Size of the descriptor in bytes (18)|
|bDescriptorType||1||1||Constant||Device Descriptor (0x01)|
|bcdUSB||2||2||BCD||USB spec version compiled with|
|bDeviceClass||4||1||Class||Code, assigned by USB org (0 means "Look at interface descriptors", common value is 2 for CDC)|
|bDeviceSubClass||5||1||SubClass||Code, assigned by USB org (usually 0)|
|bDeviceProtocol||6||1||Protocol||Code, assigned by USB org (usually 0)|
|bMaxPacketSize||7||1||Number||Max packet size for
|idVendor||8||2||ID||16bit Vendor ID (Assigned by USB org)|
|idProduct||10||2||ID||16bit Product ID (Assigned by manufacturer)|
|bcdDevice||12||2||BCD||Device version number (same encoding as bcdUSB)|
|iManufacturer||14||1||Index||String index of manufacturer name (0 if unavailable)|
|iProduct||15||1||Index||String index of product name (0 if unavailable)|
|iSerialNumber||16||1||Index||String index of device serial number (0 if unavailable)|
|bNumConfigurations||17||1||Number||Count of configurations the device has.|
bcdDevicefields is interesting too. It is of the form
MMis the major number,
mmthe minor. So USB2.0 is encoded as
0x0200, USB1.1 as
0x0110etc. If the device version is 17.36 then that'd be
0x1736. Other fields of note are
bDeviceClasswhich can be
0meaning that interfaces will specify their classes, and
idProductwhich between them form the primary way for the specific USB device to be identified. The
Indexfields are indices into a string table which we'll look at later. For now it's enough to know that wherever a string index is needed,
0can be provided to mean "no string here". The last field is
bNumConfigurationsand this indicates the number of ways in which this device might function. A USB device can provide any number of these configurations, though typically only one is provided. If the host wishes to switch between configurations then it will have to effectively entirely quiesce and reset the device. The next kind of descriptor is the configuration descriptor. This one is much shorter, but starts with the same two fields:
|Field Name||Byte start||Byte length||Encoding||Meaning|
|bLength||0||1||Number||Size of the descriptor in bytes (9)|
|bDescriptorType||1||1||Constant||Configuration Descriptor (0x02)|
|wTotalLength||2||2||Number||Size of the configuration in bytes, in total|
|bNumInterfaces||4||1||Number||The number of interfaces in this configuration|
|bConfigurationValue||5||1||Number||The value to use to select this configuration|
|iConfiguration||6||1||Index||The name of this configuration (0 for unavailable)|
|bmAttributes||7||1||Bitmap||Attributes field (see below)|
|bMaxPower||8||1||Number||Maximum bus power this configuration will draw (in 2mA increments)|
bmAttributesfield which tells the host some useful information. Bit 7 must be set, bit 6 is set if the device would be self-powered in this configuration, bit 5 indicates that the device would like to be able to wake the host from sleep mode, and bits 4 to 0 must be unset. The
bMaxPowerfield is interesting because it encodes the power draw of the device (when set to this configuration). USB allows for up to 100mA of draw per device when it isn't yet configured, and up to 500mA when configured. The value may be used to decide if it's sensible to configure a device if the host is in a low power situation. Typically this field will be set to
50to indicate the nominal 100mA is fine, or
250to request the full 500mA. Finally, the
wTotalLengthfield is interesting because it tells the host the total length of this configuration, including all the interface and endpoint descriptors which make it up. With this field, the host can allocate enough RAM to fetch the entire configuration descriptor block at once, simplifying matters dramatically for it. Each configuration has one or more interfaces. The interfaces group some endpoints together into a logical function. For example a configuration for a multifunction scanner/fax/printer might have an interface for the scanner function, one for the fax, and one for the printer. Endpoints are not shared among interfaces, so when building this table, be careful. Next, logically, come the interface descriptors:
|Field Name||Byte start||Byte length||Encoding||Meaning|
|bLength||0||1||Number||Size of the descriptor in bytes (9)|
|bDescriptorType||1||1||Constant||Interface Descriptor (0x04)|
|bInterfaceNumber||2||1||Number||The number of the interface|
|bAlternateSetting||3||1||Number||The interface alternate index|
|bNumEndpoints||4||1||Number||The number of endpoints in this interface|
|bInterfaceClass||5||1||Class||The interface class (USB Org defined)|
|bInterfaceSubClass||6||1||SubClass||The interface subclass (USB Org defined)|
|bInterfaceProtocol||7||1||Protocol||The interface protocol (USB Org defined)|
|iInterface||8||1||Index||The name of the interface (or 0 if not provided)|
bInterfaceNumberis used by the host to indicate this interface when sending messages, and the
bAlternateSettingis a way to vary interfaces. Two interfaces with the came
bAlternateSettings can be switched between (like configurations, but) without resetting the device. Hopefully the rest of this descriptor is self-evident by now. The next descriptor kind is endpoint descriptors:
|Field Name||Byte start||Byte length||Encoding||Meaning|
|bLength||0||1||Number||Size of the descriptor in bytes (7)|
|bDescriptorType||1||1||Constant||Endpoint Descriptor (0x05)|
|bEndpointAddress||2||1||Endpoint||Endpoint address (see below)|
|bmAttributes||3||1||Bitmap||Endpoint attributes (see below)|
|wMaxPacketSize||4||2||Number||Maximum packet size this endpoint can send/receive|
|bInterval||6||1||Number||Interval for polling endpoint (in frames)|
bEndpointAddressis a 4 bit endpoint number (so there're 16 endpoint indices) and a bit to indicate
OUT. Bit 7 is the direction marker and bits 3 to 0 are the endpoint number. This means there are 32 endpoints in total, 16 in each direction, 2 of which are reserved (
OUT0) giving 30 endpoints available for interfaces to use in any given configuration. The
bmAttributesbitmap covers the transfer type of the endpoint (more below), and the
bIntervalis an interval measured in frames (1ms for low or full speed, 125 s in high speed).
bIntervalis only valid for some endpoint types. The final descriptor kind is for the strings which we've seen indices for throughout the above. String descriptors have two forms:
|Field Name||Byte start||Byte length||Encoding||Meaning|
|bLength||0||1||Number||Size of the descriptor in bytes (variable)|
|bDescriptorType||1||1||Constant||String Descriptor (0x03)|
|wLangID||2||2||Number||Language code zero (e.g. 0x0409 for en_US)|
|wLangID[n]||4..||2||Number||Language code n ...|
0) is that of a series of language IDs supported by the device. The device may support any number of languages. When the host requests a string descriptor, it will supply both the index of the string and also the language id it desires (from the list available in string descriptor zero). The host can tell how many language IDs are available simply by dividing bLength by 2 and subtracting 1 for the two header bytes. And for string descriptors of an index greater than zero:
|Field Name||Byte start||Byte length||Encoding||Meaning|
|bLength||0||1||Number||Size of the descriptor in bytes (variable)|
|bDescriptorType||1||1||Constant||String Descriptor (0x03)|
|bString||2..||..||Unicode||The string, in "unicode" format|
bDescriptorTypefields which can be checked and memory allocated. Then a request for
bLengthbytes can be sent to retrieve the entire string descriptor.
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.
buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.
orchprovidercontains packages of different orchestrators such as kubernetes and nomad.
typesprovides all the generic types related to orchestrator.
pkgcontains packages like nethelper, util etc.
volumescontain packages related to volume provisioner and profiles.
buildscripts docker command docs example scripts config templates
buildscripts docker cmd docs lib api v1 artifacts docker docker-compose k8s config flaghelper loghelper mockit etcmayaserver nethelper orchprovider k8s nomad profile orchprovider volumeprovisioner server util volumeprovisioner jiva proposals
buildscripts docker command docs example orchprovider k8s v1 nomad v1 pkg nethelper util scripts config templates types v1 profile orchestrator volumes profile volumeprovisioner provisioner jiva
buildscripts docker cmd docs lib artifacts docker docker-compose k8s config flaghelper loghelper server proposals