Search Results: "jon"

2 August 2020

Holger Levsen: 20200802-debconf4

DebConf4 This tshirt is 16 years old and from DebConf4. Again, I should probably wash it at 60 celcius for once... DebConf4 was my 2nd DebConf and took place in Porto Alegre, Brasil. Like many DebConfs, it was a great opportunity to meet people: I remember sitting in the lobby of the venue and some guy asked me what I did in Debian and I told him about my little involvements and then asked him what he was doing, and he told me he wanted to become involved in Debian again, after getting distracted away. His name was Ian Murdock... DebConf4 also had a very cool history session in the hallway track (IIRC, but see below) with Bdale Garbee, Ian Jackson and Ian Murdock and with a young student named Biella Coleman busy writing notes. That same hallway also saw the kickoff meeting of the Debian Women project, though sadly today http://tinc.debian.net ("there's no cabal") only shows an apache placeholder page and not a picture of that meeting. DebCon4 was also the first time I got a bit involved in preparing DebConf, together with Jonas Smedegaard I've set up some computers there, using FAI. I had no idea that this was the start of me contributing to DebConfs for text ten years. And of course I also saw some talks, including one which I really liked, which then in turn made me notice there were no people doing video recordings, which then lead to something... I missed the group picture of this one. I guess it's important to me to mention it because I've met very wonderful people at this DebConf... (some mentioned in this post, some not. You know who you are!) Afterwards some people stayed in Porto Alegre for FISL, where we saw Lawrence Lessing present Creative Commons to the world for the first time. On the flight back I sat next to a very friendly guy from Poland and we talked almost the whole flight and then we never saw each other again, until 15 years later in Asia... Oh, and then, after DebConf4, I used IRC for the first time. And stayed in the #debconf4 IRC channel for quite some years :) Finally, DebConf4 and more importantly FISL, which was really big (5000 people?) and after that, the wizard of OS conference in Berlin (which had a very nice talk about Linux in different places in the world, illustrating the different states of 'first they ignore you, then they laugh at you, then they fight you, then you win'), made me quit my job at a company supporting Windows- and Linux-setups as I realized I'd better start freelancing with Linux-only jobs. So, once again, my life would have been different if I would not have attended these events! Note: yesterdays post about DebConf3 was thankfully corrected twice. This might well happen to this post too! :)

31 July 2020

Jonathan Carter: Free Software Activities for 2020-07

Here are my uploads for the month of July, which is just a part of my free software activities, I ll try to catch up on the rest in upcoming posts. I haven t indulged in online conferences much over the last few months, but this month I attended the virtual editions of Guadec 2020 and HOPE 2020. HOPE isn t something I knew about before and I enjoyed it a lot, you can find their videos on archive.org.

Debian Uploads 2020-07-05: Sponsor backport gamemode-1.5.1-5 for Debian buster-backports. 2020-07-06: Sponsor package piper (0.5.1-1) for Debian unstable (mentors.debian.net request). 2020-07-14: Upload package speedtest-cli (2.0.2-1+deb10u1) to Debian buster (Closes: #940165, #965116). 2020-07-15: Upload package calamares (3.2.27-1) to Debian unstable. 2020-07-15: Merge MR#1 for gnome-shell-extension-dash-to-panel. 2020-07-15: Upload package gnome-shell-extension-dash-to-panel (38-1) to Debian unstable. 2020-07-15: Upload package gnome-shell-extension-disconnect-wifi (25-1) to Debian unstable. 2020-07-15: Upload package gnome-shell-extension-draw-on-your-screen (6.1-1) to Debian unstable. 2020-07-15: Upload package xabacus (8.2.8-1) to Debian unstable. 2020-07-15: Upload package s-tui (1.0.2-1) to Debian unstable. 2020-07-15: Upload package calamares-settings-debian (10.0.2-1+deb10u2) to Debian buster (Closes: #934503, #934504). 2020-07-15: Upload package calamares-settings-debian (10.0.2-1+deb10u3) to Debian buster (Closes: #959541, #965117). 2020-07-15: Upload package calamares-settings-debian (11.0.2-1) to Debian unstable. 2020-07-19: Upload package bluefish (2.2.11+svn-r8872-1) to Debian unstable (Closes: #593413, #593427, #692284, #730543, #857330, #892502, #951143). 2020-07-19: Upload package bundlewrap (4.0.0-1) to Debian unstable. 2020-07-20: Upload package bluefish (2.2.11+svn-r8872-1) to Debian unstable (Closes: #965332). 2020-07-22: Upload package calamares (3.2.27-1~bpo10+1) to Debian buster-backports. 2020-07-24: Upload package bluefish (2.2.11_svn-r8872-3) to Debian unstable (Closes: #965944).

28 July 2020

Jonathan Carter: Free Software Activities for 2020-06

Hmm, this is the latest I ve posted my monthly updates yet (nearly by a month!). June was both crazy on the incoming side, and at the same time I just wasn t that productive (at least since then I caught up a lot). In theory, lockdown means that I spend less time in traffic, in shops or with friends and have more time to do stuff, in practice I go to bed later and later and waste more time watching tv shows and playing mobile games. A cycle that I have at least broken free from since June.

Debian Package Uploads 2020-06-04: Upload package btfs (2.21-1) to Debian unstable. 2020-06-04: Upload package gnome-shell-extension-disconnect-wifi (24-1) to Debian unstable. 2020-06-18: Sponsor package gamemode (1.5.1-5) for Debian unstable (Games team request). 2020-06-21: Upload package calamares (3.2.26-1) to Debian unstable. 2020-06-21: Upload package s-tui (1.0.1-1) to Debian unstable. 2020-06-29: Sponsor package libinih (48-1~bpo10+1) for Debian buster-backports. 2020-06-30: Upload packge calamares (3.2.26-1~bpo10+1) to Debian buster-backports. 2020-06-30: Upload package toot (0.27.0-1) to Debian unstable. 2020-06-30: Upload package calamares (3.2.26.1-1) to Debian unstable.

21 July 2020

Jonathan Dowland: FlashFloppy OLED display

This is the tenth part in a series of blog posts. The previous post was Amiga floppy recovery project: what next?. The whole series is available here: Amiga.
Rotary encoder, OLED display and mount Rotary encoder, OLED display and mount
I haven't made any substantive progress on my Amiga floppy recovery project for a while, but I felt like some retail therapy a few days ago so I bought a rotary encoder and OLED display for the Gotek floppy disk emulator along with a 3D-printed mount for them. I'm pleased with the results! The rather undescriptive "DSKA0001" in the picture is a result of my floppy image naming scheme: the display is capable of much more useful labels such as "Lemmings", "Deluxe Paint IV", etc. The Gotek and all the new bits can now be moved inside the Amiga A500's chassis.

15 July 2020

Jonathan Dowland: Lockdown music

Last Christmas, to make room for a tree, I dis-assembled my hifi unit and (temporarily, I thought) lugged my hi-fi and records up to the study.
Power, Corruption & Lies
I had been thinking about expanding the amount of storage I had for hifi and vinyl, perhaps moving from 2x2 storage cubes to 2x3, although Ikea don't make a 2x3 version of their stalwart vinyl storage line, the Kallax. I had begun exploring other options, both at Ikea and other places. Meanwhile, I re-purposed my old Expedit unit as storage for my daughter's Sylvanian Families. Under Lockdown, I've spent a lot more time in my study, and so I've set up the hifi there. It turns out I have a lot more opportunity to enjoy the records up here, during work, and I've begun to explore some things which I haven't listened to in a long time, or possibly ever. I thought I'd start keeping track of some of them. Power, Corruption and Lies is not something rarely listened to. It's steadily become my favourite New Order album. When I came across this copy (Factory, 1983), it was in pristine condition, but it now bears witness to my (mostly careful) use. There's now a scratch somewhere towards the end of the first track Age of Consent which causes my turntable to loop. By some good fortune the looping point is perfectly aligned to a bar. I don't always notice it straight away. This record rarely makes it back down from the turntable to where it's supposed to live.

6 July 2020

Jonathan Dowland: Review: Roku Express

I don't generally write consumer reviews, here or elsewhere; but I have been so impressed by this one I wanted to mention it. For Holly's birthday this year, taking place under Lockdown, we decided to buy a year's subscription to "Disney+". Our current TV receiver (A Humax Freesat box) doesn't support it so I needed to find some other way to get it onto the TV. After a short bit of research, I bought the "Roku Express" streaming media player. This is the most basic streamer that Roku make, bottom of their range. For a little bit more money you can get a model which supports 4K (although my TV obviously doesn't: it, and the basic Roku, top out at 1080p) and a bit more gets you a "stick" form-factor and a Bluetooth remote (rather than line-of-sight IR). I paid 20 for the most basic model and it Just Works. The receiver is very small but sits comfortably next to my satellite receiver-box. I don't have any issues with line-of-sight for the IR remote (and I rely on a regular IR remote for the TV itself of course). It supports Disney+, but also all the other big name services, some of which we already use (Netflix, YouTube BBC iPlayer) and some of which we didn't, since it was too awkward to access them (Google Play, Amazon Prime Video). It has now largely displaced the FreeSat box for accessing streaming content because it works so well and everything is in one place. There's a phone App that remote-controls the box and works even better than the physical remote: it can offer a full phone-keyboard at times when you need to input text, and can mute the TV audio and put it out through headphones attached to the phone if you want. My aging Plasma TV suffers from burn-in from static pictures. If left paused for a duration the Roku goes to a screensaver that keeps the whole frame moving. The FreeSat doesn't do this. My Blu Ray player does, but (I think) it retains some static elements.

26 June 2020

Jonathan Dowland: Our Study

This is a fairly self-indulgent post, sorry! Encouraged by Evgeni, Michael and others, given I'm spending a lot more time at my desk in my home office, here's a picture of it:
Fisheye shot of my home office Fisheye shot of my home office
Near enough everything in the study is a work in progress. The KALLAX behind my desk is a recent addition (under duress) because we have nowhere else to put it. Sadly I can't see it going away any time soon. On the up-side, since Christmas I've had my record player and collection in and on top of it, so I can listen to records whilst working. The arm chair is a recent move from another room. It's a nice place to take some work calls, serving as a change of scene, and I hope to add a reading light sometime. The desk chair is some Ikea model I can't recall, which is sufficient, and just fits into the desk cavity. (I'm fairly sure my Aeron, inaccessible elsewhere, would not fit.) I've had this old mahogany, leather-topped desk since I was a teenager and it's a blessing and a curse. Mostly a blessing: It's a lovely big desk. The main drawback is it's very much not height adjustable. At the back is a custom made, full-width monitor stand/shelf, a recent gift produced to specification by my Dad, a retired carpenter. On the top: my work Thinkpad T470s laptop, geared more towards portable than powerful (my normal preference), although for the forseeable future it's going to remain in the exact same place; an Ikea desk lamp (I can't recall the model); a 27" 4K LG monitor, the cheapest such I could find when I bought it; an old LCD Analog TV, fantastic for vintage consoles and the like. Underneath: An Alesis Micron 2 octave analog-modelling synthesizer; various hubs and similar things; My Amiga 500. Like Evgeni, my normal keyboard is a ThinkPad Compact USB Keyboard with TrackPoint. I've been using different generations of these styles of keyboards for a long time now, initially because I loved the trackpoint pointer. I'm very curious about trying out mechanical keyboards, as I have very fond memories of my first IBM Model M buckled-spring keyboard, but I haven't dipped my toes into that money-trap just yet. The Thinkpad keyboard is rubber-dome, but it's a good one. Wedged between the right-hand bookcases are a stack of IT-related things: new printer; deprecated printer; old/spare/play laptops, docks and chargers; managed network switch; NAS.

18 June 2020

Gunnar Wolf: On masters and slaves, whitelists and blacklists...

LWN published today yet another great piece of writing, Loaded terms in free software. I am sorry, the content will not be immediately available to anybody following at home, as LWN is based on a subscription model But a week from now, the article will be open for anybody to read. Or you can ask me (you most likely can find my contact addresses, as they are basically everywhere) for a subscriber link, I will happily provide it. In consonance with the current mood that started with the killing of George Floyd and sparked worldwide revolts against police brutality, racism (mostly related to police and law enforcement forces, but social as well) and the like, the debate that already started some months ago in technical communities has re-sparked: We have many terms that come with long histories attached to them, and we are usually oblivious to their obvious meaning. We? Yes, we, the main users and creators of technology. I never felt using master and slave to refer to different points of a protocol, bus, clock or whatever (do refer to the Wikipedia article for a fuller explanation) had any negative connotations but then again, those terms have never tainted my personal family. That is, I understand I speak from a position of privilege. A similar although less heated issue goes around the blacklist and whitelist terms, or other uses that use white to refer to good, law-abiding citizens, and black to refer to somewhat antisocial uses (i.e. the white hat and black hat hackers). For several years, this debate has been sparking and dying off. Some important changes have been made Particularly, in 2017 the Internet Software Consortium started recommending Primary and Secondary, Python dropped master/slave pairs after a quite thorough and deep review throughout 2018, GitHub changed the default branch from master to main earlier this week. The Internet Engineering Task Force has a draft (that lapsed and thus sadly didn t become an RFC, but still, is archived), Terminology, Power and Oppressive Language that lists suggested alternatives:
There are also many other relationships that can be used as metaphors, Eglash s research calls into question the accuracy of the master-slave metaphor. Fortunately, there are ample alternatives for the master-slave relationship. Several options are suggested here and should be chosen based on the pairing that is most clear in context:
  • Primary-secondary
  • Leader-follower
  • Active-standby
  • Primary-replica
  • Writer-reader
  • Coordinator-worker
  • Parent-helper
I ll add that I think we Spanish-speakers are not fully aware of the issue s importance, because the most common translation I have seen for master/slave is maestro/esclavo: Maestro is the word for teacher (although we do keep our slaves in place). But think whether it sounds any worse if you refer to device pairs, or members of a database high-availability cluster, or whatever as Amo and Esclavo. It does sound much worse I cannot add much of value to this debate. I am just happy issues like this are being recognized and dealt with. If the topic interests you, do refer to the LWN article! Some excrepts: I consider the following to be the core of Jonathan Corbet s writeup:
Recent events, though, have made it clear even to those of us who were happy to not question this view that the story of slavery and the wider racist systems around it is not yet finished. There are many people who are still living in the middle of it, and it is not a nice place to be. We are not so enlightened as we like to think we are. If there is no other lesson from the events of the last few weeks, we should certainly take to heart the point that we need to be listening to the people who have been saying, for many years, that they are still suffering. If there are people who are telling us that terms like slave or blacklist are a hurtful reminder of the inequities that persist in our society, we need to accept that as the truth and act upon it. Etymological discussions on what, say, master really means may be interesting, but they miss the point and are irrelevant to this discussion.
Part of a comment by user yokem_55:
Often, it seems to me that the replacement words are much more descriptive and precise than the old language. Allowlist is far more obviously a list of explicitly authorized entities than whitelist . Mainline has a more obvious meaning of a core stream of development than master . The benefit of moving past this language is more than just changing cultural norms, it s better, more precise communication across the board.
Another spot-on comment, by user alan:
From my perspective as a Black American male, I think that it s nice to see people willing to see and address racism in various spheres. I am concerned that some of these steps will be more performative than substantial. Terminology changes in software so as to be more welcoming is a nice thing. Ensuring that oppressed minorities have access to the tools and resources to help reduce inequity and ensuring equal protection under the laws is better. We ll get there one day I m sure. The current ask is much simpler, its just to stop randomly killing and terrorizing us. Please and thank you.
So Maybe the protests of this year caught special notoriety because the society is reacting after (or during, for many of us) the lockdown. In any case, I hope for their success in changing the planet s culture of oppression.

Comments Tomas Janousek 2020-06-19 10:04:32 +0200 In the blog post On masters and slaves, whitelists and blacklists you claim that GitHub changed the default branch from master to main earlier this week but I don t think that change is in effect yet. When you create a repo, the default branch is still named master . Gunnar Wolf 2020-06-19 11:52:30 -0500 Umh, seems you are right. Well, what can I say? I m reporting only what I have been able to find / read Now, given that said master branch does not carry any Git-specific meaning and is just a commonly used configuration I hope people start picking it up. No, I have not renamed master branches in any of my repos but intend to do so soonish. Tomas Janousek 2020-06-19 20:01:52 +0200 Yeah, don t worry. I just find it sad that so much inaccurate news is spreading from a single CEO tweet, and I wanted to help stop that. I m sure some change will happen eventually, but until it does, we shouldn t speak about it in the past tense. :-)

11 June 2020

Markus Koschany: My Free Software Activities in May 2020

Welcome to gambaru.de. Here is my monthly report (+ the first week in June) that covers what I have been doing for Debian. If you re interested in Java, Games and LTS topics, this might be interesting for you. Debian Games
Debian Java Misc Debian LTS This was my 51. month as a paid contributor and I have been paid to work 25 hours on Debian LTS, a project started by Rapha l Hertzog. In that time I did the following: ELTS Extended Long Term Support (ELTS) is a project led by Freexian to further extend the lifetime of Debian releases. It is not an official Debian project but all Debian users benefit from it without cost. The current ELTS release is Debian 7 Wheezy . This was my 24. month and I have been paid to work 9,25 hours on ELTS. Thanks for reading and see you next time.

10 June 2020

Jonathan Dowland: Template Haskell and Stream-processing programs

I've written about what Template Haskell is, and given an example of what it can be used for, it's time to explain why I was looking at it in the context of my PhD work. Encoding stream-processing programs StrIoT is an experimental distributed stream-processing system that myself and others are building in order to explore our research questions. A user of StrIoT writes a stream-processing program, using a set of 8 functional operators provided for the purpose. A simple example is
streamFn :: Stream Int -> Stream Int
streamFn = streamFilter (<15)
         . streamFilter (>5)
         . streamMap (*2)
Our system is distributed: we take a stream-processing program and partition it into sub-programs, which are distributed to and run on separate nodes (perhaps cloud instances, or embedded devices like Raspberry Pis etc.). In order to do that, we need to be able to manipulate the stream-processing program as data. We've initially opted for a graph data-structure, with the vertices in the graph defined as
data StreamVertex = StreamVertex
      vertexId   :: Int
    , operator   :: StreamOperator
    , parameters :: [String]
    , intype     :: String
    , outtype    :: String
      deriving (Eq,Show)
A stream-processing program encoded this way, equivalent to the first example
path [ StreamVertex 0 Map    ["(*2)"]  "Int" "Int"
     , StreamVertex 1 Filter ["(>5)"]  "Int" "Int"
     , StreamVertex 2 Filter ["(<15)"] "Int" "Int"
     ]
We can easily manipulate instances of such types, rewrite them, partition them and generate code from them. Unfortunately, this is quite a departure from the first simple code example from the perspective of a user writing their program. Template Haskell gives us the ability to manipulate code as a data structure, and also to inspect names to gather information about them (their type, etc.). I started looking at TH to see if we could build something where the user-supplied program was as close to that first case as possible. TH limitations There are two reasons that we can't easily manipulate a stream-processing definition written as in the first example. The following expressions are equivalent, in some sense, but are not equal, and so yield completely different expression trees when quasi-quoted:
[  streamFilter (<15) . streamFilter (>5) . streamMap (*2)  ]
  \s -> streamFilter (<15) (streamFilter (>5) (streamMap (*2) s))  ]
[  streamMap (*2) >>> streamFilter (>5) >>> streamFilter (<15)  ]
[  \s -> s & streamMap (*2) & streamFilter (>5) & streamFilter (<15)  ]
[  streamFn  ] -- a named expression, defined outside the quasi-quotes
In theory, reify can give you the definition of a function from its name, but in practice it doesn't, because this was never implemented. So at the very least we would need to insist that a user included the entirety of a stream-processing program within quasi-quotes, and not split it up into separate bits, with some bits defined outside the quotes and references within (as in the last case above). We would probably have to insist on a consistent approach for composing operators together, such as always use (.) and never >>>, &, etc. which is limiting. Incremental approach After a while ruminating on this, and before moving onto something else, I thought I'd try approaching it from the other side. Could I introduce some TH into the existing approach, and improve it? The first thing I've tried is to change the parameters field to TH's ExpQ, meaning the map instance example above would be
StreamVertex 0 Map [ [  (*2)  ] ] "Int" "Int"
I worked this through. It's an incremental improvement ease and clarity for the user writing a stream-processing program. It catches a class of programming bugs that would otherwise slip through: the expressions in the brackets have to be syntactically valid (although they aren't type checked). Some of the StrIoT internals are also much improved, particularly the logical operator. Here's an excerpt from a rewrite rule that involves composing code embedded in strings, dealing with all the escaping rules and hoping we've accounted for all possible incoming expression encodings:
let f' = "(let f = ("++f++"); p = ("++p++"); g = ("++g++") in\
         \ \\ (a,b) v -> (f a v, if p v a then g b v else b))"
    a' = "("++a++","++b++")"
    q' = "(let p = ("++p++"); q = ("++q++") in \\v (y,z) -> p v y && q v z)"
And the same section after, manipulating ExpQ types:
let f' = [  \ (a,b) v -> ($(f) a v, if $(p) v a then $(g) b v else b)  ]
    a' = [  ($(a), $(b))  ]
    q' = [  \v (y,z) -> $(p) v y && $(q) v z  ]
I think the code-generation part of StrIoT could be radically refactored to take advantage of this change but I have not made huge inroads into that. Next steps This is, probably, where I am going to stop. This work is very interesting to me but not the main thrust of my research. But incrementally improving the representation gave me some ideas of what I could try next: The type would have collapsed down to
data StreamVertex = StreamVertex
      vertexId   :: Int
    , opAndParams :: ExpQ
      deriving (Eq,Show)
Example instances might be
StreamVertex 0 [  streamMap (*2)  ]
StreamVertex 1 [  streamExpand  ]
StreamVertex 2 [  streamScan (\c _ -> c+1) 0  ]
The vertexId field is a bit of wart, but we require that due to the graph data structure that we are using. A change there could eliminate it, too. By this point we are not that far away from where we started, and certainly much closer to the "pure" function application in the very first example.

Jonathan Dowland: template haskell

I've been meaning to write more about my PhD work for absolutely ages, but I've held myself back by wanting to try and keep a narrative running through the blog posts. That's not realistic for a number of reasons so I'm going to just write about different aspects of things without worrying about whether they make sense in the context of recent blog posts or not. Part of what I am doing at the moment is investigating Template Haskell to see whether it would usefully improve our system implementation. Before I write more about how it might apply to our system, I'll first write a bit about Template Haskell itself. Template Haskell (TH) is a meta-programming system: you write programs that are executed at compile time and can output code to be spliced into the parent program. The approach used by TH is really nice: you perform your meta-programming in real first-class Haskell, and it integrates really well with the main program. TH provides two pairs of special brackets. Oxford brackets surrounding any Haskell expression cause the whole expression to be replaced by the result of parsing the expression an expression tree which can be inspected and manipulated by the main program:
[  \x -> x + 1  ]
The expression data-type is a series of mutually-recursive data types that represent the complete Haskell grammar. The top-level is Exp, for expression, which has constructors for the different expression types. The above lambda expression is represented as
LamE [VarP x_1]
    (InfixE (Just (VarE x_1))
            (VarE GHC.Num.+)
            (Just (LitE (IntegerL 1))))
Such expressions can be pattern-matched against, constructed, deconstructed etc just like any other data type. The other bracket type performs the opposite operation: it takes an expression structure and splices it into code in the main program, to be compiled as normal:
 > 1 + $( litE (IntegerL 1) )
2
The two are often intermixed, sometimes nested to several levels. What follows is a typical beginner TH meta-program. The standard function fst operators on a 2-tuple and returns the first value. It cannot operate on a tuple of a different valence. However, a meta-program can generate a version of fst specialised for an n-tuple of any n:
genfst n = do
    xs <- replicateM n (newName "x")
    let ntup = tupP (map varP xs)
    [  \ $(ntup) ->  $(varE (head xs))  ]
Used like so
 > $(genfst 2) (1,2)
1
 > $(genfst 3) ('a','b','c')
'a'
 > :t $(genfst 10)
$(genfst 10) :: (a, b, c, d, e, f, g, h, i, j) -> a
That's a high-level gist of how you can use TH. I've skipped over a lot of detail, in particular an important aspect relating to scope and naming, which is key to the problem I am exploring at the moment. Oxford brackets and slice brackets do not operate directly on the simple Exp data-type, but upon an Exp within the Q Monad:
 > :t [  1  ]
[  1  ] :: ExpQ
ExpQ is a synonym for Q Exp. Eagle-eyed Haskellers will have noticed that genfst above was written in terms of some Monad. And you might also have noticed the case discrepancy between the constructor types VarE (Etc) and varE, tupP, varP used in that function definition. These are convenience functions that wrap the relevant constructor in Q. The point of the Q Monad is (I think) to handle name scoping, and avoid unintended name clashes. Look at the output of these simple expressions, passed through runQ:
 > runQ [  \x -> x  ]
LamE [VarP x_1] (VarE x_1)
 > runQ [  \x -> x  ]
LamE [VarP x_2] (VarE x_2)
Those x are not the same x in the context they are evaluated (a GHCi session). And that's the crux of the problem I am exploring. More in a later blog post!

3 June 2020

Jonathan Dowland: using Template Haskell to generate boilerplate

Here's a practical example of applying Template Haskell to reduce the amount of boilerplate code that is otherwise required. I wrote the below after following this excellent blog post by Matt Parsons. This post will be much higher-level, read Matt's blog for the gorier details. Liquorice Liquorice is a toy project of mine from a few years ago that lets you draw 2D geometric structures similar to LOGO. Liquorice offers two interfaces: pure functions that operate on an explicit Context (the pen location: existing lines, etc.), and a second "stateful" interface where the input and output are handled in the background. I prefix the pure ones P. and the stateful ones S. in this blog post for clarity. The stateful interface can be much nicer to use for larger drawings. Compare example8b.hs, written in terms of the pure functions, and the stateful equivalent example8.hs. The majority of the stateful functions are "wrapped" versions of the pure functions. For example, the pure function P.step takes two numbers and moves the pen forward and sideways. Its type signature is
P.step :: Int -> Int -> Context -> Context
Here's the signature and implementation of the stateful equivalent:
S.step :: Int -> Int -> State Context ()
S.step x y = modify (P.step x y)
Writing these wrapped functions for the 29 pure functions is boilerplate that can be generated automatically with Template Haskell. Generating the wrapper functions Given the Name of a function to wrap, we construct an instance of FunD, the TH data-type representing a function definition. We use the base name of the incoming function as the name for the new one.
mkWrap fn = do
     
    let name = mkName (nameBase fn)
    return $ FunD name [ Clause (map VarP args) (NormalB rhs) [] ]
To determine how many arguments the wrapper function needs to accept, we need to determine the input function's arity. We use Template Haskell's reify function to get type information about the function, and derive the arity from that. Matt Parson's covers this exactly in his blog.
info    <- reify fn
let ty   = (\(VarI _ t _ ) -> t) info
let n    = arity ty - 1
args    <- replicateM n (newName "arg")
We can use the list "args" directly in the clause part of the function definition, as the data-type expects a list. For the right-hand side, we need to convert from a list of arguments to function application. That's a simple left-fold:
-- mkFnApp f [a,b,c] => ((f a) b) c => f a b c
mkFnApp = foldl (\e -> appE e . varE)
rhs     <- [  modify $(mkFnApp (varE fn) args) :: State Context ()  ]
We use TH's oxford brackets for the definition of rhs. This permits us to write real Haskell inside the brackets, and get an expression data-type outside them. Within we have a splice (the $( )), which does the opposite: the code is evaluated at compile time and generates an Exp that is then converted into the equivalent Haskell code and spliced into place. Finally, we need to apply the above to a list of Names. Sadly, we can't get at the list of exported names from a Module automatically. There is an open request for a TH extension for this. In the meantime, we export a list of the functions to wrap from the Pure module and operate on that
import Liquorice.Pure
wrapPureFunctions = mapM mkWrap pureFns
Finally, we 'call' wrapPureFunctions at the top level in our state module and Template Haskell splices all the function definitions into place. The final code ended up only around 30 lines of code, and saved about the same number of lines of boilerplate. But in doing this I noticed some missing functions, and it will pay dividends if more pure functions are added. Limitations The current implementation has one significant limitation: it cannot handle higher-order functions. An example of a pure higher-order function is place, which moves the pen, performs an operation, and then moves it back:
P.place :: Int -> Int -> (Context -> Context) -> Context -> Context
Wrapping this is not sufficient because the higher-order parameter has the pure function signature Context -> Context. If we wrapped it, the stateful version of the function would accept a pure function as the parameter, but you would expect it to accept another stateful function. To handle these, at a minimum we would need to detect the function arguments that have type Context -> Context and replace them with State Context (). The right-hand side of the wrapped function would also need to do more work to handle wrapping and unwrapping the parameter. I haven't spent much time thinking about it but I'm not sure that a general purpose wrapper would work for all higher-order functions. For the time being I've just re-implemented the half-dozen of them.

1 June 2020

Jonathan Carter: Free Software Activities for 2020-05

I would say that this was a crazy month, but with everything ever escalating, does that even mean anything anymore? I lost track of tracking my activities in the second half of the month, and I m still not very good at logging the soft stuff , that is, things like non-technical work but that also takes up a lot of time, but will continue to work on it.
Towards the end of the month I spent a huge amount of time on MiniDebConf Online, I m glad it all worked out, and will write a seperate blog entry on that. Thank you again to everyone for making it a success! I m also moving DPL activities to the DPL blog, so even though it s been a busy month in the free software world my activity log here will look somewhat deceptively short this month MiniDebConf Online 2020-05-06: Help prepare initial CfP mail. 2020-05-06: Process some feedback regarding accessibility on Jitsi.

Debian Packaging 2020-05-02: Upload package gnome-shell-extension-workspaces-to-dock (53-1) to Debian unstable. 2020-05-02: Upload package tetzle (2.1.6-1) to Debian unstable. 2020-05-06: Upload package bundlewrap (3.9.0-1) to Debian unstable. 2020-05-06: Accept MR#1 for connectagram. 2020-05-06: Upload package connectagram (1.2.11-1) to Debian unstable. 2020-05-07: Upload package gnome-shell-extension-multi-monitors (20-1) to Debian unstable (Closes: #956169). 2020-05-07: Upload package tanglet (1.5.6-1) to Debian unstable. 2020-05-16: Upload package calamares (3.2.24-1) to Debian unstable. 2020-05-16: Accept MR#1 for tuxpaint-config. 2020-05-16: Accept MR#7 for debian-live. 2020-05-18: Upload package bundlewrap (3.10.0) to Debian unstable.

Debian Mentoring 2020-05-02: Sponsor package gamemode (1.5.1-3) (Games team request). 2020-05-16: Sponsor package gamemode (1.5.1-4) (Games team request).

Paul Wise: FLOSS Activities May 2020

Focus This month I didn't have any particular focus. I just worked on issues in my info bubble.

Changes

Issues

Review

Administration
  • nsntrace: talk to upstream about collaborative maintenance
  • Debian: deploy changes, debug issue with GPS markers file generation, migrate bls/DUCK from alioth-archive to salsa
  • Debian website: ran map cron job, synced mirrors
  • Debian wiki: approve accounts, ping folks with bouncing email

Communication

Sponsors The apt-offline work and the libfile-libmagic-perl backports were sponsored. All other work was done on a volunteer basis.

31 May 2020

Jonathan Dowland: Golf Peaks

tricky tricky
This one was a bit of a surprise hit. Golf Peaks is a Golf-themed puzzle game. The objective is to get the golf ball into the hole, obviously, but how you do that is by choosing from a set of fixed moves (e.g., move one square; jump one square; etc.) and a direction. This works well with my eldest daughter. She takes one joy-con and I take the other. I typically am responsible for direction, and she hits 'A' to move the ball. We discuss which move to make, which has been a good test of her numeracy.

Jonathan McDowell: OpenOCD snapshot uploaded to Debian experimental

One of the things I maintain in Debian is OpenOCD. I say maintain, but it s so far required very little work, as it s been 3 years since a release (0.10.0). I ve talked about doing a git snapshot package for some time (I have an email from last DebConf in my inbox about it, and that wasn t the first time someone had asked), but never got around to it. Spurred on by some moves towards a 0.11.0 release I ve built a recent snapshot and uploaded it to the experimental suite in Debian. Of particular interest is the support for more recent architectures that this brings - ARMv8/aarch64 and RISC-V being the big ones, but also MIPS64 and various other ARM improvements. I no longer have access to Xilinx Zynq or Mellanox Bluefield platforms to test against so I ve just done some some basic tests with a Sheevaplug and BusPirate/STM32F103, but those worked just fine. Builds should hopefully happen shortly. Enjoy!

29 May 2020

Gunnar Wolf: Heads up Online MiniDebConf is Online

I know most Debian people know about this already But in case you don t follow the usual Debian communications channels, this might interest you! Given most of the world is still under COVID-19 restrictions, and that we want to work on Debian, given there is no certainty as to what the future holds in store for us Our DPL fearless as they always are had the bold initiative to make this weekend into the first-ever miniDebConf Online (MDCO)! miniDebConf Online So, we are already halfway through DebCamp (which means, you can come and hang out with us in the debian.social DebCamp Jitsi lounge, where some impromptu presentations might happen (or not). Starting tomorrow morning (11AM UTC), we will have a quite interesting set of talks. I am reproducing the schedule here:

Saturday 2020.05.30
Time (UTC) Speaker Talk
11:00 - 11:10 MDCO team members Hello + Welcome
11:30 - 11:50 Wouter Verhelst Extrepo
12:00 - 12:45 JP Mengual Debian France, trust european organization
13:00 - 13:20 Arnaud Ferraris Bringing Debian to mobile phones, one package at a time
13:30 - 15:00 Lunch Break A chance for the teams to catch some air
15:00 - 15:45 JP Mengual The community team, United Nations Organizations of Debian?
16:00 - 16:45 Christoph Biedl Clevis and tang - overcoming the disk unlocking problem
17:00 - 17:45 Antonio Terceiro I m a programmer, how can I help Debian

Sunday 2020.05.31
Time (UTC) Speaker Talk
11:00 - 11:45 Andreas Tille The effect of Covid-19 on the Debian Med project
12:00 - 12:45 Paul Gevers BoF: running autopkgtest for your package
13:00 - 13:20 Ben Hutchings debplate: Build many binary packages with templates
13:30 - 15:00 Lunch break A chance for the teams to catch some air
15:00 - 15:45 Holger Levsen Reproducing bullseye in practice
16:00 - 16:45 Jonathan Carter Striving towards excellence
17:00 - 17:45 Delib* Organizing Peer-to-Peer Debian Facilitation Training
18:00 - 18:15 MDCO team members Closing
  • subject to confirmation

Timezone Remember this is an online event, meant for all of the world! Yes, the chosen times seem quite Europe-centric (but they are mostly a function of the times the talk submitters requested). Talks are 11:00 18:00UTC, which means, 06:00 13:00 Mexico (GMT-5), 20:00 03:00 Japan (GMT+9), 04:00 11:00 Western Canada/USA/Mexico (GMT-7) and the rest of the world, somewhere in between. (No, this was clearly not optimized for our dear usual beer team. Sorry! I guess we need you to be fully awake at beertime!)

[update] Connecting! Of course, I didn t make it clear at first how to connect to the Online miniDebConf, silly me!
  • The video streams are available at: https://video.debconf.org/
  • Suggested: tune in to the #minidebconf-online IRC channel in OFTC.
That should be it. Hope to see you there! (Stay home, stay safe )

Gunnar Wolf: Heads up Online MiniDebConf is Online

I know most Debian people know about this already But in case you don t follow the usual Debian communications channels, this might interest you! Given most of the world is still under COVID-19 restrictions, and that we want to work on Debian, given there is no certainty as to what the future holds in store for us Our DPL fearless as they always are had the bold initiative to make this weekend into the first-ever miniDebConf Online (MDCO)! miniDebConf Online So, we are already halfway through DebCamp (which means, you can come and hang out with us in the debian.social DebCamp Jitsi lounge, where some impromptu presentations might happen (or not). Starting tomorrow morning (11AM UTC), we will have a quite interesting set of talks. I am reproducing the schedule here:

Saturday 2020.05.30
Time (UTC) Speaker Talk
11:00 - 11:10 MDCO team members Hello + Welcome
11:30 - 11:50 Wouter Verhelst Extrepo
12:00 - 12:45 JP Mengual Debian France, trust european organization
13:00 - 13:20 Arnaud Ferraris Bringing Debian to mobile phones, one package at a time
13:30 - 15:00 Lunch Break A chance for the teams to catch some air
15:00 - 15:45 JP Mengual The community team, United Nations Organizations of Debian?
16:00 - 16:45 Christoph Biedl Clevis and tang - overcoming the disk unlocking problem
17:00 - 17:45 Antonio Terceiro I m a programmer, how can I help Debian

Sunday 2020.05.31
Time (UTC) Speaker Talk
11:00 - 11:45 Andreas Tille The effect of Covid-19 on the Debian Med project
12:00 - 12:45 Paul Gevers BoF: running autopkgtest for your package
13:00 - 13:20 Ben Hutchings debplate: Build many binary packages with templates
13:30 - 15:00 Lunch break A chance for the teams to catch some air
15:00 - 15:45 Holger Levsen Reproducing bullseye in practice
16:00 - 16:45 Jonathan Carter Striving towards excellence
17:00 - 17:45 Delib* Organizing Peer-to-Peer Debian Facilitation Training
18:00 - 18:15 MDCO team members Closing
  • subject to confirmation

Timezone Remember this is an online event, meant for all of the world! Yes, the chosen times seem quite Europe-centric (but they are mostly a function of the times the talk submitters requested). Talks are 11:00 18:00UTC, which means, 06:00 13:00 Mexico (GMT-5), 20:00 03:00 Japan (GMT+9), 04:00 11:00 Western Canada/USA/Mexico (GMT-7) and the rest of the world, somewhere in between. (No, this was clearly not optimized for our dear usual beer team. Sorry! I guess we need you to be fully awake at beertime!)

[update] Connecting! Of course, I didn t make it clear at first how to connect to the Online miniDebConf, silly me!
  • The video streams are available at: https://video.debconf.org/
  • Suggested: tune in to the #minidebconf-online IRC channel in OFTC.
That should be it. Hope to see you there! (Stay home, stay safe )

Keith Packard: picolibc-string-float

Float/String Conversion in Picolibc Exact conversion between strings and floats seems like a fairly straightforward problem. There are two related problems:
  1. String to Float conversion. In this case, the goal is to construct the floating point number which most closely approximates the number represented by the string.
  2. Float to String conversion. Here, the goal is to generate the shortest string which, when fed back into the String to Float conversion code, exactly reproduces the original value.
When linked together, getting from float to string and back to float is a round trip , and an exact pair of algorithms does this for every floating point value. Solutions for both directions were published in the proceedings of the ACM SIGPLAN 1990 conference on Programming language design and implementation, with the string-to-float version written by William Clinger and the float-to-string version written by Guy Steele and Jon White. These solutions rely on very high precision integer arithmetic to get every case correct, with float-to-string requiring up to 1050 bits for the 64-bit IEEE floating point format. That's a lot of bits. Newlib Float/String Conversion The original newlib code, written in 1998 by David M. Gay, has arbitrary-precision numeric code for these functions to get exact results. However, it has the disadvantages of performing numerous memory allocations, consuming considerable space for the code, and taking a long time for conversions. The first disadvantage, using malloc during conversion, ended up causing a number of CVEs because the results of malloc were not being checked. That's bad on all platforms, but especially bad for embedded systems where reading and writing through NULL pointers may have unknown effects. Upstream newlib applied a quick fix to check the allocations and call abort. Again, on platforms with an OS, that at least provides a way to shut down the program and let the operating environment figure out what to do next. On tiny embedded systems, there may not be any way to log an error message or even restart the system. Ok, so we want to get rid of the calls to abort and have the error reported back through the API call which caused the problem. That's got two issues, one mere technical work, and another mere re-interpretation of specifications. Let's review the specification issue. The libc APIs involved here are: Input: Output: Scanf and printf are both documented to set errno to ENOMEM when they run out of memory, but none of the other functions takes that possibility into account. So we'll make some stuff up and hope it works out: Now, looking back at the technical challenge. That's a simple matter of inserting checks at each allocation, or call which may result in an allocation, and reporting failure back up the call stack, unwinding any intermediate state to avoid leaking memory. Testing Every Possible Allocation Failure There are a lot of allocation calls in the newlib code. And the call stack can get pretty deep. A simple visual inspection of the code didn't seem sufficient to me to validate the allocation checking code. So I instrumented malloc, making it count the number of allocations and fail at a specific one. Now I can count the total number of allocations done over the entire test suite run for each API involved and then run the test suite that many times, failing each allocation in turn and checking to make sure we recover correctly. By that, I mean: There were about 60000 allocations to track, so I ran the test suite that many times, which (with the added malloc tracing enabled) took about 12 hours. Bits Pushed to the Repository With the testing complete, I'm reasonably confident that the code is now working, and that these CVEs are more completely squashed. If someone is interested in back-porting the newlib fixes upstream to newlib, that would be awesome. It's not completely trivial as this part of picolibc has diverged a bit due to the elimination of the reent structure. Picolibc's Tinystdio Float/String Conversion Picolibc contains a complete replacement for stdio which was originally adopted from avr libc. That's a stdio implementation designed to run on 8-bit Atmel processors and focuses on very limited memory use and small code size. It does this while maintaining surprisingly complete support for C99 printf and scanf support. However, it also does this without any arbitrary precision arithmetic, which means it doesn't get the right answer all of the time. For most embedded systems, this is usually a good trade off -- floating point input and output are likely to be largely used for diagnostics and debugging, so mostly correct answers are probably sufficient. The original avr-libc code only supports 32-bit floats, as that's all the ABI on those processors has. I extended that to 64-, 80- and 128- bit floats to cover double and long double on x86 and RISC-V processors. Then I spent a bunch of time adjusting the code to get it to more accurately support C99 standards. Tinystdio also had strtod support, but it was missing ecvt, fcvt and gcvt. For those, picolibc was just falling back to the old newlib code, which introduced all of the memory allocation issues we've just read about. Fixing that so that tinystdio was self-contained and did ecvt, fcvt and gcvt internally required writing those functions in terms of the float-to-string primitives already provided in tinystdio to support printf. gcvt is most easily supported by just calling sprintf. Once complete, the default picolibc build, using tinystdio, no longer does any memory allocation for float/string conversions.

5 May 2020

Jonathan Dowland: Introducing Red Hat UBI OpenJDK runtime images

At Red Hat we've just shipped Universal Base Image (UBI) build images for OpenJDK. Download them with your favourite container manager, e.g.:
podman pull registry.access.redhat.com/ubi8/openjdk-8
podman pull registry.access.redhat.com/ubi8/openjdk-11
UBI, announced a year ago, is an initiative where you can obtain, share and build upon official Red Hat container images without needing a Red Hat subscription. Unlike something like CentOS, they aren't modified in any way (e.g. to remove branding), they're exactly the same base images that Red Hat products are built upon, composed entirely of Open Source software. Your precise rights are covered in the EULA. I work on the Red Hat OpenJDK container images, which are designed primarily for use with OpenShift. We've been based upon the RHEL base images since inception. Although our containers are open source (of course), we haven't been able to distribute the binary images more widely than to Red Hat customers, until now. These are based upon the "minimal" flavour base image and add an OpenJDK runtime and development packages (either JDK8 or JDK11, as per the image name) as well as Maven and OpenShift integration scripts that allow the image to be used with Source To Image. If you give these a try, please let us know what you think! Comment here, on twitter, or file Issues or Pull Requests in the GitHub project.

Next.