Jonathan Carter: DebConf 20 Online
RuboCop::Packaging.RuboCop's latest version.lib/ and gemspec file.Puppet6.php-dasprid-enum and php-bacon-baconqrcode for William and ruby-unparser, ruby-morpher, and ruby-path-exapander for Cocoa.
In May, I got selected as a Google Summer of Code student for Debian again! \o/docs/ directory, which is hosted at https://docs.rubocop.org/rubocop-packaging.gem install rubocop-packaging, but also via apt install ruby-rubocop-packaging.
Packaging extension reports as offensive .git ls-files in gemspec.RuboCop and RuboCop::Packaging.RuboCop to v0.89..rubocop.yml and add RuboCop::Packaging as a development dependency.RuboCop and RuboCop::RSpec version to the latest.git ls-files in gemspec.
:wq for today.
Dealing with the void during MiniDebConf Online #1
Between 28 and 31 May this year, we set out to create our first ever online MiniDebConf for Debian. Many people have been meaning to do something similar for a long time, but it just didn t work out yet. With many of us being in lock down due to COVID-19, and with the strong possibility looming that DebConf20 might have had to become an online event, we rushed towards organising the first ever Online MiniDebConf and put together some form of usable video stack for it.
I could go into all kinds of details on the above, but this post is about a bug that lead to a pretty nifty feature for DebConf20. The tool that we use to capture Jitsi calls is called Jibri (Jitsi Broadcasting Infrustructure). It had a bug (well, bug for us, but it s an upstream feature) where Jibri would hang up after 30s of complete silence, because it would assume that the call has ended and that the worker can be freed up again. This would result in the stream being ended at the end of every talk, so before the next talk, someone would have to remember to press play again in their media player or on the video player on the stream page. Hrmph.
Easy solution on the morning that the conference starts? I was testing a Debian Live image the night before in a KVM and thought that I might as well just start a Jitsi call from there and keep a steady stream of silence so that Jibri doesn t hang up.
It worked! But the black screen and silence on stream was a bit eery. Because this event was so experimental in nature, and because we were on such an incredibly tight timeline, we opted not to seek sponsors for this event, so there was no sponsors loop that we d usually stream during a DebConf event. Then I thought Ah! I could just show the schedule! .
DebConf20 Moves Online For DebConf, we usually show a sponsors loop in between sessions. It s great that we give our sponsors visibility here, but in reality people see the sponsors loop and think Talk over! and then they look away. It s also completely silent and doesn t provide any additional useful information. I was wondering how I could take our lessons from MDCO#1 and integrate our new tricks with the sponsors loop. That is, add the schedule, time, some space to type announcements on the screen and also add some loopable music to it. I used OBS before in making my videos, and like the flexibility it provides when working with scenes and sources. A scene is what you would think of as a screen or a document with its own collection of sources or elements. For example, a scene might contain sources such as a logo, clock, video, image, etc. A scene can also contain another scene. This is useful if you want to contain a banner or play some background music that is shared between scenes.
The Loopy Loop Music The two mini albums that mostly played during the first few days were just a copy and paste from the MDCO#1 music, which was:
For shoutout tracks, that were later used in the loop too (because it became a bit monotonous), most of the tracks came from freepd.com:
Further Switch game recommendations
Morphite is a first-person space exploration game, with a very distinctive
aesthetic, which reminds me a little bit of No Man's Sky. This is a fairly
child-friendly game of exploration and discovery. It also reminds me a little
bit of Frontier: First Encounters, the second sequel to Elite.
![[pic]](https://jmtd.net/log/morphite/morphite-switch-screenshot05.jpg)
![[pic]](https://jmtd.net/log/morphite/morphite-switch-screenshot06.jpg)
Often, when I mention how things work in the interactive theorem prover [Isabelle/HOL] (in the following just Isabelle 1) to people with a strong background in functional programming (whether that means Haskell or Coq or something else), I cause confusion, especially around the issue of what is a function, are function total and what is the business with undefined. In this blog post, I want to explain some these issues, aimed at functional programmers or type theoreticians.
Note that this is not meant to be a tutorial; I will not explain how to do these things, and will focus on what they mean.
f :: a b between two types a and b (the function arrow in Isabelle is , not ), then by definition of what it means to be a function in HOL whenever I have a value x :: a, then the expression f x (i.e. f applied to x) is a value of type b. Therefore, and without exception, every Isabelle function is total.
In particular, it cannot be that f x does not exist for some x :: a. This is a first difference from Haskell, which does have partial functions like
spin :: Maybe Integer -> Bool
spin (Just n) = spin (Just (n+1))spin Nothing nor the expression spin (Just 42) produce a value of type Bool: The former raises an exception ( incomplete pattern match ), the latter does not terminate. Confusingly, though, both expressions have type Bool.
Because every function is total, this confusion cannot arise in Isabelle: If an expression e has type t, then it is a value of type t. This trait is shared with other total systems, including Coq.
Did you notice the emphasis I put on the word is here, and how I deliberately did not write evaluates to or returns ? This is because of another big source for confusion:
a b in functional programming is an algorithm that, given a value of type a, calculates (returns, evaluates to) a value of type b.a b in math (or Isabelle) associates with each value of type a a value of type b.definition foo :: "(nat real) real" where
"foo seq = (if convergent seq then lim seq else 0)"fun plus :: "nat nat nat" where
"plus 0 m = m"
"plus (Suc n) m = Suc (plus n m)"which is clearly a description of a computation. But to Isabelle, the above readsplusis a function that analyses its first argument. If that is0, then it returns the second argument. Otherwise, it calls itself with the predecessor of the first argument and increases the result by one.
plus is a binary function on natural numbers, and it satisfies the following two equations:
And in fact, it is not so much Isabelle that reads it this way, but rather the fun command, which is external to the Isabelle logic. The fun command analyses the given equations, constructs a non-recursive definition of plus under the hood, passes that to Isabelle and then proves that the given equations hold for plus.
One interesting consequence of this is that different specifications can lead to the same functions. In fact, if we would define plus' by recursing on the second argument, we d obtain the the same function (i.e. plus = plus' is a theorem, and there would be no way of telling the two apart).
fun command can only construct plus in a way that the equations hold if it passes a termination check very much like Fixpoint in Coq.
But while the termination check of Fixpoint in Coq is a deep part of the basic logic, in Isabelle it is simply something that this particular command requires for its internal machinery to go through. At no point does a termination proof of the function exist as a theorem inside the logic. And other commands may have other means of defining a function that do not even require such a termination argument!
For example, a function specification that is tail-recursive can be turned in to a function, even without a termination proof: The following definition describes a higher-order function that iterates its first argument f on the second argument x until it finds a fixpoint. It is completely polymorphic (the single quote in 'a indicates that this is a type variable):
partial_function (tailrec)
fixpoint :: "('a 'a) 'a 'a"
where
"fixpoint f x = (if f x = x then x else fixpoint f (f x))"f with ( x. x-1), we can prove that it will always return 0:
lemma "fixpoint ( n . n - 1) (n::nat) = 0"
by (induction n) (auto simp add: fixpoint.simps)partial_function (option) collatz :: "nat nat list option"
where "collatz n =
(if n = 1 then Some [n]
else if even n
then do ns <- collatz (n div 2); Some (n # ns)
else do ns <- collatz (3 * n + 1); Some (n # ns) )"fixpoint be a total function? What is fixpoint ( n. n+1)? What if we run collatz n for a n where the Collatz sequence does not reach 1?3 We will come back to that question after a little detour
undefined :: 'aarbitrary or, even better, unknown.
Since undefined can be instantiated at any type, we can instantiate it for example at bool, and we can observe an important fact: undefined is not an extra value besides the usual ones . It is simply some value of that type, which is demonstrated in the following lemma:
lemma "undefined = True undefined = False" by autoundefined for sure:
lemma "undefined = ()" by autofromJust function:
fun fromSome :: "'a option 'a" where
"fromSome (Some x) = x"fun (albeit with a warning), and the generated function fromSome behaves exactly as specified: when applied to Some x, it is x. The term fromSome None is also a value of type 'a, we just do not know which one it is, as the specification does not address that.
So fromSome None behaves just like undefined above, i.e. we can prove
lemma "fromSome None = False fromSome None = True" by autofun constOrId :: "bool bool" where
"constOrId True = True"
lemma "constOrId = ( _.True) constOrId = ( x. x)"
by (metis (full_types) constOrId.simps)fixpoint above. Clearly, the function seen as a functional program is not total: When passed the argument ( n. n + 1) or ( b. b) it will loop forever trying to find a fixed point.
But Isabelle functions are not functional programs, and the definitions are just specifications. What does the specification say about the case when f has no fixed-point? It states that the equation fixpoint f x = fixpoint f (f x) holds. And this equation has a solution, for example fixpoint f _ = undefined.
Or more concretely: The specification of the fixpoint function states that fixpoint ( b. b) True = fixpoint ( b. b) False has to hold, but it does not specify which particular value (True or False) it should denote any is fine.
f and get a function out of that? But rest assured: That is not the case. For example, no Isabelle command allows you define a function bogus :: () nat with the equation bogus () = Suc (bogus ()), because this equation does not have a solution.
We can actually prove that such a function cannot exist:
lemma no_bogus: " bogus. bogus () = Suc (bogus ())" by simpnot_bogus () = not_bogus () is just fine )
undefined is not a separate, recognizable value, but rather simply an unknown one, there is no way of stating that A function result is not specified .
Here is an example that demonstrates this: Two partial functions (one with not all cases specified, the other one with a self-referential specification) are indistinguishable from the total variant:
fun partial1 :: "bool unit" where
"partial1 True = ()"
partial_function (tailrec) partial2 :: "bool unit" where
"partial2 b = partial2 b"
fun total :: "bool unit" where
"total True = ()"
"total False = ()"
lemma "partial1 = total partial2 = total" by auto () and partial2 = total. We have done that to verify some of HLint s equations.
fixpoint, which is not possible in Coq.
There is currently ongoing work about verified code generation, where the code equations are reflected into a deep embedding of HOL in Isabelle that would allow explicit termination proofs.
undefined in Coq
This section is speculative, and an invitation for discussion.
Coq already distinguishes between types used in programs (Set) and types used in proofs Prop.
Could Coq ensure that every t : Set is non-empty? I imagine this would require additional checks in the Inductive command, similar to the checks that the Isabelle command datatype has to perform4, and it would disallow Empty_set.
If so, then it would be sound to add the following axiom
Axiom undefined : forall (a : Set), a._ undefined. Would it help with non-obviously terminating functions? Would it allow a Coq command Tailrecursive that accepts any tailrecursive function without a termination check?
fun, the constructions by datatype are not part of the logic, but create a type definition from more primitive notions that is isomorphic to the specified data type. ErrorDocument 400 directive pointing to a cgid script
in worker mode).
Still, it was a serious regression and I found a part of the nasty
long patch we worked on back then that was faulty, and introduced a
small fix to correct that. The proposed package unfortunately
didn't yield any feedback, and I can only assume it will work okay for
people. The result is the DLA-841-2 upload which fixes the
regression. I unfortunately didn't have time to work on the remaining
CVEs affecting apache2 in LTS at the time of writing.
The name comes from what snakes and other animals do to "create a new snake": they shed their skin. This is not so appropriate for snakes, as it's just a way to rejuvenate their skin, but is especially relevant for anthropods since then "ecdysis" may be associated with a metamorphosis:It was an interesting exercise to go back into older projects and factor out interesting code. The process is not complete yet, as there are older projects I'm still curious in reviewing. A bunch of that code could also be factored into upstream project and maybe even the Python standard library. In short, this is stuff I keep on forgetting how to do: a properEcdysis is the moulting of the cuticle in many invertebrates of the clade Ecdysozoa. Since the cuticle of these animals typically forms a largely inelastic exoskeleton, it is shed during growth and a new, larger covering is formed. The remnants of the old, empty exoskeleton are called exuviae. WikipediaSo this project is metamorphosed into others when the documentation templates, code examples and so on are reused elsewhere. For that reason, the license is an unusally liberal (for me) MIT/Expat license. The name also has the nice property of being absolutely unpronounceable, which makes it unlikely to be copied but easy to search online.
setup.py config, some fancy argparse extensions and so on. Instead
of having to remember where I had written that clever piece of code, I
now shove it in the crazy chaotic project where I can find it again in
the future.
set builtin: for small
items, it is basically as fast as a bloom filter. Of course, when
the item size grows larger, its memory usage explodes, but in this
case it turned out to be sufficient and bloom filter completely
overkill and confusing.
Oh, and thanks to those efforts, I got admitted in the beetbox
organization on GitHub! I am not sure what I will do with that
newfound power: I was just scratching an itch, really. But hopefully
I'll be able to help here and there in the future as well.
Earlier today, I demonstrated how to work with coinduction in the theorem provers Isabelle, Coq and Agda, with a very simple example. This reminded me of a discussion I had in Karlsruhe with my then colleague Denis Lohner: If coinduction is the dual of induction, why do the induction principles look so different? I like what we observed there, so I d like to share this.
The following is mostly based on my naive understanding of coinduction based on what I observe in the implementation in Isabelle. I am sure that a different, more categorial presentation of datatypes (as initial resp. terminal objects in some category of algebras) makes the duality more obvious, but that does not necessarily help the working Isabelle user who wants to make sense of coninduction.
datatype 'a list = nil cons (hd : 'a) (tl : "'a list")P nil ( x xs. P xs P (cons x xs)) xs. P xscodatatype 'a llist = lnil lcons (hd : 'a) (tl : "'a llist")( xs ys.
R xs ys' (xs = lnil) = (ys = lnil)
(xs lnil ys' lnil
hd xs = hd ys R (tl xs) (tl ys)))
( xs ys. R xs ys xs = ys)R between lists which implies that either both lists are empty or both lists are nonempty, and furthermore if both are non-empty, that they have the same head and tails related by R, then any two lists related by R are actually equal.
If you think of the infinte list as a series of states of a computer program, then this is nothing else than a bisimulation.
So we have two proof principles, both of which make intuitive sense. But how are they related? They look very different! In one, we have a predicate P, in the other a relation R, to point out just one difference.
datatype declaration automatically creates a relator:
rel_list :: ('a 'b bool) 'a list 'b list boolrel_list R xs ys is that xs and ys have the same shape (i.e. length), and that the corresponding elements are pairwise related by R. You might have defined this relation yourself at some time, and if so, you probably introduced it as an inductive predicate. So it is not surprising that the following induction principle characterizes this relation:
Q nil nil
( x xs y ys. R x y Q xs ys Q (cons x xs) (cons y ys))
( xs ys rel_list R xs ys Q xs ys)Q xs ys (P xs xs = ys) and R x y (x = y), then we obtain exactly that. In that sense, the relation induction is a generalization of the normal induction.
codatatype declaration introduces a function
rel_llist :: ('a 'b bool) 'a llist 'b llist bool( xs ys.
R xs ys (xs = lnil) = (ys = lnil)
(xs lnil ys lnil
Q (hd xs) (hd ys) R (tl xs) (tl ys)))
( xs ys R xs ys rel_llist Q xs ys)Q with equality, which turns rel_llist Q into equality on the lists, and you have the theorem above.
( xs ys.
R xs ys (xs = lnil) = (ys = lnil)
(xs lnil ys lnil
Q (hd xs) (hd ys) R (tl xs) (tl ys)))
= case analysis (the other two cases are vacuously true)
( xs ys.
xs = lnil ys = lnil
R xs ys (xs = lnil) = (ys = lnil)
(xs lnil ys lnil
Q (hd xs) (hd ys) R (tl xs) (tl ys)))
( xs ys.
xs lnil ys lnil
R xs ys (xs = lnil) = (ys = lnil)
(xs lnil ys lnil
Q (hd xs) (hd ys) R (tl xs) (tl ys)))
= simplification
( xs ys. xs = lnil ys = lnil R xs ys
( x xs y ys. R (cons x xs) (cons y ys) (Q x y R xs ys))
= more rewriting
R nil nil
( x xs y ys. Q x y R xs ys R (cons x xs) (cons y ys))
Home office minister Amber Rudd recently suggested British firms should publish details about how many foreign people they employ and in which positions. She argued this is necessary to help boost funding for training local people.
If that is such a brilliant idea, why hasn't it worked for the Premier League? It is a matter of public knowledge how many foreigners play football in England's most prestigious division, so why hasn't this caused local clubs to boost training budgets for local recruits? After all, when you consider that England hasn't won a World Cup since 1966, what have they got to lose?
All this racism, it's just not cricket. Or is it? One of the most remarkable cricketers to play for England in recent times, Kevin Pietersen, dubbed "the most complete batsman in cricket" by The Times and "England's greatest modern batsman" by the Guardian, was born in South Africa. In the five years he was contracted to the Hampshire county team, he only played one match, because he was too busy representing England abroad. His highest position was nothing less than becoming England's team captain.
Are the British superior to every other European citizen?
One of the implications of the rhetoric coming out of London these days is that the British are superior to their neighbours, entitled to have their cake and eat it too, making foreigners queue up at Paris' Gare du Nord to board the Eurostar while British travelers should be able to walk or drive into European countries unchallenged.
This superiority complex is not uniquely British, you can observe similar delusions are rampant in many of the places where I've lived, including Australia, Switzerland and France. America's Donald Trump has taken this style of politics to a new level.
Look in the mirror Theresa May: after British 10-year old schoolboys Robert Thompson and Jon Venables abducted, tortured, murdered and mutilated 2 year old James Bulger in 1993, why not have all British schoolchildren fingerprinted and added to the police DNA database? Why should "security" only apply based on the country where people are born, their religion or skin colour?
In fact, after Brexit, people like Venables and Thompson will remain in Britain while a Dutch woman, educated at Cambridge and with two British children will not. If that isn't racism, what is?
Running foreigner's off the roads
Theresa May has only been Prime Minister for less than a year but she has a history of bullying and abusing foreigners in her previous role in the Home Office. One example of this was a policy of removing driving licenses from foreigners, which has caused administrative chaos and even taken away the licenses of many people who technically should not have been subject to these regulations anyway.
Shouldn't the DVLA (Britain's office for driving licenses) simply focus on the competence of somebody to drive a vehicle? Bringing all these other factors into licensing creates a hostile environment full of mistakes and inconvenience at best and opportunities for low-level officials to engage in arbitrary acts of racism and discrimination.
Of course, when you are taking your country on the road to nowhere, who needs a driving license anyway?
What does "maximum control" over other human beings mean to you?
The new British PM has said she wants "maximum control" over immigrants. What exactly does "maximum control" mean? Donald Trump appears to be promising "maximum control" over Muslims, Hitler sought "maximum control" over the Jews, hasn't the whole point of the EU been to avoid similar situations from ever arising again?
This talk of "maximum control" in British politics has grown like a weed out of the UKIP. One of their senior figures has been linked to kidnappings and extortion, which reveals a lot about the character of the people who want to devise and administer these policies. Similar people in Australia aspire to jobs in the immigration department where they can extort money out of people for getting them pushed up the queue. It is no surprise that the first member of Australia's parliament ever sent to jail was put there for obtaining bribes and sexual favours from immigrants. When Nigel Farage talks about copying the Australian immigration system, he is talking about creating jobs like these for his mates.
Even if "maximum control" is important, who really believes that a bunch of bullies in Westminster should have the power to exercise that control? Is May saying that British bosses are no longer competent to make their own decisions about who to employ or that British citizens are not reliable enough to make their own decisions about who they marry and they need a helping hand from paper-pushers in the immigration department?
Echoes of the Third Reich
Most people associate acts of mass murder with the Germans who lived in the time of Adolf Hitler. These are the stories told over and and over again in movies, books and the press.
Look more closely, however, and it appears that the vast majority of Germans were not in immediate contact with the gas chambers. Even Gobels' secretary writes that she was completely oblivious to it all. Many people were simply small cogs in a big bad machine. The clues were there, but many of them couldn't see the big picture. Even if they did get a whiff of it, many chose not to ask questions, to carry on with their comfortable lives.
Today, with mass media and the Internet, it is a lot easier for people to discover the truth if they look, but many are still reluctant to do so.
Consider, for example, the fingerprint scanners installed in British post offices and police stations to fingerprint foreigners and criminals (as if they have something in common). If all the post office staff refused to engage in racist conduct the fingerprint scanners would be put out of service. Nonetheless, these people carry on, just doing their job, just following orders. It was through many small abuses like this, rather than mass murder on every street corner, that Hitler motivated an entire nation to serve his evil purposes.
Technology like this is introduced in small steps: first it was used for serious criminals, then anybody accused of a crime, then people from Africa and next it appears they will try and apply it to all EU citizens remaining in the UK.
How will a British man married to a French woman explain to their children that mummy has to be fingerprinted by the border guard each time they return from vacation?
The Nazis pioneered biometric technology with the tracking numbers branded onto Jews. While today's technology is electronic and digital, isn't it performing the same function?
There is no middle ground between "soft" and "hard" brexit
An important point for British citizens and foreigners in the UK to consider today is that there is no compromise between a "soft" Brexit and a "hard" Brexit. It is one or the other. Anything less (for example, a deal that is "better" for British companies and worse for EU citizens) would imply that the British are a superior species and it is impossible to imagine the EU putting their stamp on such a deal. Anybody from the EU who is trying to make a life in the UK now is playing a game of Russian Roulette - sure, everything might be fine if it morphs into "soft" Brexit, but if Theresa May has her way, at some point in your life, maybe 20 years down the track, you could be rounded up by the gestapo and thrown behind bars for a parking violation. There has already been a five-fold increase in the detention of EU citizens in British concentration camps and they are using grandmothers from Asian countries to refine their tactics for the efficient removal of EU citizens. One can only wonder what type of monsters Theresa May has been employing to run such inhumane operations.
This is not politics
Edmund Burke's quote "The only thing necessary for the triumph of evil is for good men to do nothing" comes to mind on a day like today. Too many people think it is just politics and they can go on with their lives and ignore it. Barely half the British population voted in the referendum. This is about human beings treating each other with dignity and respect. Anything less is abhorrent and may well come back to bite.
FOSDEM is one of the largest (5,000+ hackers!) gatherings of Free Software contributors in the world and happens each February in Brussels (Belgium, Europe).
Once again, one of the tracks will be the Desktops DevRoom (formerly known as CrossDesktop DevRoom ), which will host Desktop-related talks.
We are now inviting proposals for talks about Free/Libre/Open-source Software on the topics of Desktop development, Desktop applications and interoperability amongst Desktop Environments. This is a unique opportunity to show novel ideas and developments to a wide technical audience.
Topics accepted include, but are not limited to:
If my presentation is accepted for FOSDEM, I hereby agree to license all recordings, slides, and other associated materials under the Creative Commons Attribution Share-Alike 4.0 International License. Sincerely, <NAME>.If you want us to stop the recording in the Q & A part (should you have one), please tell us. We can do that but only for the Q & A part. More information The official communication channel for the Desktops DevRoom is its mailing list desktops-devroom@lists.fosdem.org. Use this page to manage your subscription: https://lists.fosdem.org/listinfo/desktops-devroom Organization The Desktops DevRoom 2017 is managed by a team representing the most notable open desktops:
Here is my monthly update covering a large part of what I have been doing in the free software world (previously):
Here is my monthly update covering a large part of what I have been doing in the free software world (previously):
As nearly anyone who's worked with me will attest to, I've long since
touted nedbat's talk
Pragmatic Unicode, or, How do I stop the pain?
as one of the most foundational talks, and required watching for all programmers.
The reason is because netbat hits on something bigger - something more
fundamental than how to handle Unicode -- it's how to handle data which is
relative.
For those who want the TL;DR, the argument is as follows:
Facts of Life:
Windows-1252, UTF-8, Latin-1, or EBCDIC. What's
a poem to me is terminal garbage to you.
Over the years, hacks have evolved. We have
magic numbers,
and plain ole' hacks to just guess based on the content. Of course, like
all good computer programs, this has lead to its fair share of hilarious
bugs,
and there's nothing stopping files from (validly!) being multiple things at the
same time.
Like many things, it's all in the eye of the beholder.
Timezones
Just like Unicode, this is a word that can put your friendly neighborhood
programmer into a series of profanity laden tirades. Go find one in the wild,
and ask them about what they think about timezone handling bugs they've seen.
I'll wait. Go ahead.
Rants are funny things. They're fun to watch. Hilarious to give. Sometimes
just getting it all out can help. They can tell you a lot about the true
nature of problems.
It's funny to consider the isomorphic nature of Unicode rants and Timezone
rants.
I don't think this is an accident.
U n i c o d e timezone Sandwich
Ned's Unicode Sandwich applies -- As early as we can, in the lowest level
we can (reading from the database, filesystem, wherever!), all datetimes
must be timezone qualified with their correct timezone. Always. If you mean
UTC, say it's in UTC.
Treat any unqualified datetimes as "bytes". They're not to be trusted.
Never, never, never trust 'em. Don't
process any datetimes until you're sure they're in the right timezone.
This lets the delicious inside of your datetime sandwich handle timezones
with grace, and finally, as late as you can, turn it back into bytes
(if at all!). Treat locations as tzdb entries, and qualify datetime
objects into their absolute timezone (EST, EDT, PST, PDT)
It's not until you want to show the datetime to the user again should you
consider how to re-encode your datetime to bytes. You should think about
what flavor of bytes, what encoding -- what timezone -- should I be
encoding into?
TEST
Just like Unicode, testing that your code works with datetimes is important.
Every time I think about how to go about doing this, I think about that
one time that mjg59 couldn't book a flight
starting Tuesday from AKL, landing in HNL on Monday night, because
United couldn't book the last leg to SFO. Do you ever assume dates only go
forward as time goes on? Remember timezones.
Construct test data, make sure someone in New Zealand's
+13:45 can correctly talk with
their friends in
Baker Island's -12:00,
and that the events sort right.
Just because it's Noon on New Years Eve in England doesn't mean it's not
1 AM the next year in New Zealand. Places a few miles apart may go on Daylight
savings different days. Indian Standard Time is not even aligned on the hour
to GMT (+05:30)!
Test early, and test often. Memorize a few timezones, and challenge
your assumptions when writing code that has to do with time. Don't use
wall clocks to mean monotonic time. Remember there's a whole world out there,
and we only deal with part of it.
It's also worth remembering, as Andrew Pendleton
pointed out to me, that it's possible that a datetime isn't even unique for a
place, since you can never know if 2016-11-06 01:00:00 in America/New_York
(in the tzdb) is the first one, or second one. Storing EST or EDT along
with your datetime may help, though!
Pitfalls
Improper handling of timezones can lead to some interesting things, and failing
to be explicit (or at least, very rigid) in what you expect will lead to an
unholy class of bugs we've all come to hate. At best, you have confused
users doing math, at worst, someone misses a critical event, or our
security code fails.
I recently found what I regard to be a pretty bad
bug in apt (which David has prepared a
fix
for and is pending upload, yay! Thank you!), which boiled down to documentation
and code expecting datetimes in a timezone, but accepting any timezone, and
silently treating it as UTC.
The solution is to hard-fail, which is an interesting choice to me (as a vocal
fan of timezone aware code), but at the least it won't fail by
misunderstanding what the server is trying to communicate, and I do understand
and empathize with the situation the apt maintainers are in.
Final Thoughts
Overall, my main point is although most modern developers know how to deal
with Unicode pain, I think there is a more general lesson to learn -- namely,
you should always know what data you have, and always remember what it is.
Understand assumptions as early as you can, and always store them with the data.
-fdebug-prefix-map to the compilers options)
Propellor was recently ported to FreeBSD, by Evan Cofsky. This new feature
led me down a two week long rabbit hole to make it type safe. In particular,
Propellor needed to be taught that some properties work on Debian, others on
FreeBSD, and others on both.
The user shouldn't need to worry about making a mistake like this;
the type checker should tell them they're asking for something that can't
fly.
-- Is this a Debian or a FreeBSD host? I can't remember, let's use both package managers!
host "example.com" $ props
& aptUpgraded
& pkgUpgraded
upgraded = aptUpgraded pickOS pkgUpgraded
aptUpgraded :: Property (MetaTypes '[ 'Targeting 'OSDebian, 'Targeting 'OSBuntish ])
pkgUpgraded :: Property (MetaTypes '[ 'Targeting 'OSFreeBSD ])
' if you have not seen that before. There are some convenience
aliases and type operators, which let the same types be expressed
more cleanly:
aptUpgraded :: Property (Debian + Buntish)
pkgUpgraded :: Property FreeBSD
aptUpgraded and pkgUpgraded
will yield a metatypes that targets no OS's, since they have none in
common. So will fail to type check.
My implementation of the metatypes lists is hundreds of lines of
code, consisting entirely of types and type families. It includes a basic
implementation of singletons, and is portable back to ghc 7.6 to support
Debian stable. While it takes some contortions to support such an old
version of ghc, it's pretty awesome that the ghc in Debian stable supports
this stuff.
extending beyond targeted OS's
Before this change, Propellor's Property type had already been slightly
refined, tagging them with HasInfo or NoInfo, as described
in making propellor safer with GADTs and type families. I needed to
keep that HasInfo in the type of properties.
But, it seemed unnecessary verbose to have types like Property NoInfo Debian.
Especially if I want to add even more information to Property
types later. Property NoInfo Debian NoPortsOpen would be a real mouthful to
need to write for every property.
Luckily I now have this handy type-level list. So, I can shove more
types into it, so Property (HasInfo + Debian) is used where necessary,
and Property Debian can be used everywhere else.
Since I can add more types to the type-level list, without affecting
other properties, I expect to be able to implement type-level port
conflict detection next. Should be fairly easy to do without changing
the API except for properties that use ports.
singletons
As shown here, pickOS makes a property that
decides which of two properties to use based on the host's OS.
aptUpgraded :: Property DebianLike
aptUpgraded = property "apt upgraded" (apt "upgrade" requires apt "update")
pkgUpgraded :: Property FreeBSD
pkgUpgraded = property "pkg upgraded" (pkg "upgrade")
upgraded :: Property UnixLike
upgraded = (aptUpgraded pickOS pkgUpgraded)
describe "OS upgraded"
pickOS property combinator.
Its implementation needs to be able to look at each of the properties at
runtime, to compare the OS's they target with the actial OS of the host.
That's done by stashing a target list value inside a property. The target
list value is inferred from the type of the property, thanks to singletons,
and so does not need to be passed in to property. That saves
keyboard time and avoids mistakes.
is it worth it?
It's important to consider whether more complicated types are a net
benefit. Of course, opinions vary widely on that question in general!
But let's consider it in light of my main goals for Propellor:
foo :: Property UnixLike
foo = p requires bar
where
p = property "foo" $ do
...
p because many
different types could be combined with the bar property and all would
yield a Property UnixLike. The solution is simply to add a type signature
like p :: Property UnixLike
Since this only affects creating new properties, and not combining existing
properties (which have known types), it seems like a reasonable tradeoff.
things to improve later
There are a few warts that I'm willing to live with for now...
Currently, Property (HasInfo + Debian) is different than Property (Debian +
HasInfo), but they should really be considered to be the same type. That is, I
need type-level sets, not lists. While there's a type level sets library for
hackage, it still seems to
require a specific order of the set items when writing down a type signature.
Also, using ensureProperty, which runs one property inside the action
of another property, got complicated by the need to pass it a type witness.
foo = Property Debian
foo = property' $ \witness -> do
ensureProperty witness (aptInstall "foo")
Here is my monthly update covering a large part of what I have been doing in the free software world (previously):
FOSDEM is one of the largest gatherings of Free Software contributors in the world and happens each February in Brussels (Belgium, Europe). One of the tracks will be the Desktops DevRoom (formerly known as CrossDesktop DevRoom ), which will host Desktop-related talks.
We are now inviting proposals for talks about Free/Libre/Open-source Software on the topics of Desktop development, Desktop applications and interoperability amongst Desktop Environments. This is a unique opportunity to show novel ideas and developments to a wide technical audience.
Topics accepted include, but are not limited to:
Here is my monthly update covering a large part of what I have been doing in the free software world (previously):
I will join Lars and Tincho in stating this, and presenting a version contrary to what Norbert portraits.
I am very glad and very proud that the community I am most involved in, the Debian project, has kept its core identity over the years, at least for the slightly-over-a-decade I have been involved in it. And I am very glad and very proud that being less aggressive, more welcoming and in general more respectful to each other does not counter this.
When I joined Debian, part of the mantra chants we had is that in order to join a Free Software project you had to grow a thick skin, as sooner or later we'd all be exposed to flamefests. But, yes, the median age of the DD was way lower back then I don't have the data at hand, but IIRC I have always been close to our median. Which means we are all growing old and grumpy. But old and wiser.
A very successful, important and dear subproject to many of us is the Debian Women Project. Its original aim was, as the name shows, to try to reduce the imbalance between men and women participants in Debian IIRC back in 2004 we had 3 female DDs, and >950 male DDs. Soon, the project started morphing into pushing all of Debian to be less hostile, more open to contributions from any- and everyone (as today our diversity statement reads).
And yes, we are still a long, long, long way from reaching equality. But we have done great steps. And not just WRT women, but all of the different minorities, as well as to diverging opinions within our community. Many people don't enjoy us abiding by a code of conduct; I also find it irritating sometimes to have to abide by certain codes if we mostly know each other and know we won't be offended by a given comment... Or will we?
So, being more open and more welcoming also means being more civil. I cannot get myself to agree with Linus' quote, when he says that respect is not just given to everybody but must be earned. We should always start, and I enjoy feeling that in Debian this is becoming the norm, by granting respect to everybody And not losing it, even if things get out of hand. Thick skins are not good for communication.
In a few weeks, I will have the opportunity to offer a weekend workshop to selected and motivated high school students1 to a topic of my choice. My idea is to tell them something about logic, proofs, and the joy of searching and finding proofs, and the gratification of irrevocable truths.
While proving things on paper is already quite nice, it is much more fun to use an interactive theorem prover, such as Isabelle, Coq or Agda: You get immediate feedback, you can experiment and play around if you are stuck, and you get lots of small successes. Someone2 once called interactive theorem proving the worlds most geekiest videogame .
Unfortunately, I don t think one can get high school students without any prior knowledge in logic, or programming, or fancy mathematical symbols, to do something meaningful with a system like Isabelle, so I need something that is (much) easier to use. I always had this idea in the back of my head that proving is not so much about writing text (as in normally written proofs) or programs (as in Agda) or labeled statements (as in Hilbert-style proofs), but rather something involving facts that I have proven so far floating around freely, and way to combine these facts to new facts, without the need to name them, or put them in a particular order or sequence. In a way, I m looking for labVIEW wrestled through the Curry-Horward-isomorphism. Something like this:
A proof of implication currying
Next.