Search Results: "seb"

02 November 2017

Antoine Beaupr : October 2017 report: LTS, feed2exec beta, pandoc filters, git mediawiki

Debian Long Term Support (LTS) This is my monthly Debian LTS report. This time I worked on the famous KRACK attack, git-annex, golang and the continuous stream of GraphicsMagick security issues.

WPA & KRACK update I spent most of my time this month on the Linux WPA code, to backport it to the old (~2012) wpa_supplicant release. I first published a patchset based on the patches shipped after the embargo for the oldstable/jessie release. After feedback from the list, I also built packages for i386 and ARM. I have also reviewed the WPA protocol to make sure I understood the implications of the changes required to backport the patches. For example, I removed the patches touching the WNM sleep mode code as that was introduced only in the 2.0 release. Chunks of code regarding state tracking were also not backported as they are part of the state tracking code introduced later, in 3ff3323. Finally, I still have concerns about the nonce setup in patch #5. In the last chunk, you'll notice peer->tk is reset, to_set to negotiate a new TK. The other approach I considered was to backport 1380fcbd9f ("TDLS: Do not modify RNonce for an TPK M1 frame with same INonce") but I figured I would play it safe and not introduce further variations. I should note that I share Matthew Green's observations regarding the opacity of the protocol. Normally, network protocols are freely available and security researchers like me can easily review them. In this case, I would have needed to read the opaque 802.11i-2004 pdf which is behind a TOS wall at the IEEE. I ended up reading up on the IEEE_802.11i-2004 Wikipedia article which gives a simpler view of the protocol. But it's a real problem to see such critical protocols developed behind closed doors like this. At Guido's suggestion, I sent the final patch upstream explaining the concerns I had with the patch. I have not, at the time of writing, received any response from upstream about this, unfortunately. I uploaded the fixed packages as DLA 1150-1 on October 31st.

Git-annex The next big chunk on my list was completing the work on git-annex (CVE-2017-12976) that I started in August. It turns out doing the backport was simpler than I expected, even with my rusty experience with Haskell. Type-checking really helps in doing the right thing, especially considering how Joey Hess implemented the fix: by introducing a new type. So I backported the patch from upstream and notified the security team that the jessie and stretch updates would be similarly easy. I shipped the backport to LTS as DLA-1144-1. I also shared the updated packages for jessie (which required a similar backport) and stretch (which didn't) and those Sebastien Delafond published those as DSA 4010-1.

Graphicsmagick Up next was yet another security vulnerability in the Graphicsmagick stack. This involved the usual deep dive into intricate and sometimes just unreasonable C code to try and fit a round tree in a square sinkhole. I'm always unsure about those patches, but the test suite passes, smoke tests show the vulnerability as fixed, and that's pretty much as good as it gets. The announcement (DLA 1154-1) turned out to be a little special because I had previously noticed that the penultimate announcement (DLA 1130-1) was never sent out. So I made a merged announcement to cover both instead of re-sending the original 3 weeks late, which may have been confusing for our users.

Triage & misc We always do a bit of triage even when not on frontdesk duty, so I: I also did smaller bits of work on: The latter reminded me of the concerns I have about the long-term maintainability of the golang ecosystem: because everything is statically linked, an update to a core library (say the SMTP library as in CVE-2017-15042, thankfully not affecting LTS) requires a full rebuild of all packages including the library in all distributions. So what would be a simple update in a shared library system could mean an explosion of work on statically linked infrastructures. This is a lot of work which can definitely be error-prone: as I've seen in other updates, some packages (for example the Ruby interpreter) just bit-rot on their own and eventually fail to build from source. We would also have to investigate all packages to see which one include the library, something which we are not well equipped for at this point. Wheezy was the first release shipping golang packages but at least it's shipping only one... Stretch has shipped with two golang versions (1.7 and 1.8) which will make maintenance ever harder in the long term.
We build our computers the way we build our cities--over time, without a plan, on top of ruins. - Ellen Ullman

Other free software work This month again, I was busy doing some serious yak shaving operations all over the internet, on top of publishing two of my largest LWN articles to date (2017-10-16-strategies-offline-pgp-key-storage and 2017-10-26-comparison-cryptographic-keycards).

feed2exec beta Since I announced this new project last month I have released it as a beta and it entered Debian. I have also wrote useful plugins like the wayback plugin that saves pages on the Wayback machine for eternal archival. The archive plugin can also similarly save pages to the local filesystem. I also added bash completion, expanded unit tests and documentation, fixed default file paths and a bunch of bugs, and refactored the code. Finally, I also started using two external Python libraries instead of rolling my own code: the pyxdg and requests-file libraries, the latter which I packaged in Debian (and fixed a bug in their test suite). The program is working pretty well for me. The only thing I feel is really missing now is a retry/fail mechanism. Right now, it's a little brittle: any network hiccup will yield an error email, which are readable to me but could be confusing to a new user. Strangely enough, I am particularly having trouble with (local!) DNS resolution that I need to look into, but that is probably unrelated with the software itself. Thankfully, the user can disable those with --loglevel=ERROR to silence WARNINGs. Furthermore, some plugins still have some rough edges. For example, The Transmission integration would probably work better as a distinct plugin instead of a simple exec call, because when it adds new torrents, the output is totally cryptic. That plugin could also leverage more feed parameters to save different files in different locations depending on the feed titles, something would be hard to do safely with the exec plugin now. I am keeping a steady flow of releases. I wish there was a way to see how effective I am at reaching out with this project, but unfortunately GitLab doesn't provide usage statistics... And I have received only a few comments on IRC about the project, so maybe I need to reach out more like it says in the fine manual. Always feels strange to have to promote your project like it's some new bubbly soap... Next steps for the project is a final review of the API and release production-ready 1.0.0. I am also thinking of making a small screencast to show the basic capabilities of the software, maybe with asciinema's upcoming audio support?

Pandoc filters As I mentioned earlier, I dove again in Haskell programming when working on the git-annex security update. But I also have a small Haskell program of my own - a Pandoc filter that I use to convert the HTML articles I publish on LWN.net into a Ikiwiki-compatible markdown version. It turns out the script was still missing a bunch of stuff: image sizes, proper table formatting, etc. I also worked hard on automating more bits of the publishing workflow by extracting the time from the article which allowed me to simply extract the full article into an almost final copy just by specifying the article ID. The only thing left is to add tags, and the article is complete. In the process, I learned about new weird Haskell constructs. Take this code, for example:
-- remove needless blockquote wrapper around some tables
--
-- haskell newbie tips:
--
-- @ is the "at-pattern", allows us to define both a name for the
-- construct and inspect the contents as once
--
--   is the "empty record pattern": it basically means "match the
-- arguments but ignore the args"
cleanBlock (BlockQuote t@[Table  ]) = t
Here the idea is to remove <blockquote> elements needlessly wrapping a <table>. I can't specify the Table type on its own, because then I couldn't address the table as a whole, only its parts. I could reconstruct the whole table bits by bits, but it wasn't as clean. The other pattern was how to, at last, address multiple string elements, which was difficult because Pandoc treats spaces specially:
cleanBlock (Plain (Strong (Str "Notifications":Space:Str "for":Space:Str "all":Space:Str "responses":_):_)) = []
The last bit that drove me crazy was the date parsing:
-- the "GAByline" div has a date, use it to generate the ikiwiki dates
--
-- this is distinct from cleanBlock because we do not want to have to
-- deal with time there: it is only here we need it, and we need to
-- pass it in here because we do not want to mess with IO (time is I/O
-- in haskell) all across the function hierarchy
cleanDates :: ZonedTime -> Block -> [Block]
-- this mouthful is just the way the data comes in from
-- LWN/Pandoc. there could be a cleaner way to represent this,
-- possibly with a record, but this is complicated and obscure enough.
cleanDates time (Div (_, [cls], _)
                 [Para [Str month, Space, Str day, Space, Str year], Para _])
    cls == "GAByline" = ikiwikiRawInline (ikiwikiMetaField "date"
                                           (iso8601Format (parseTimeOrError True defaultTimeLocale "%Y-%B-%e,"
                                                           (year ++ "-" ++ month ++ "-" ++ day) :: ZonedTime)))
                        ++ ikiwikiRawInline (ikiwikiMetaField "updated"
                                             (iso8601Format time))
                        ++ [Para []]
-- other elements just pass through
cleanDates time x = [x]
Now that seems just dirty, but it was even worse before. One thing I find difficult in adapting to coding in Haskell is that you need to take the habit of writing smaller functions. The language is really not well adapted to long discourse: it's more about getting small things connected together. Other languages (e.g. Python) discourage this because there's some overhead in calling functions (10 nanoseconds in my tests, but still), whereas functions are a fundamental and important construction in Haskell that are much more heavily optimized. So I constantly need to remind myself to split things up early, otherwise I can't do anything in Haskell. Other languages are more lenient, which does mean my code can be more dirty, but I feel get things done faster then. The oddity of Haskell makes frustrating to work with. It's like doing construction work but you're not allowed to get the floor dirty. When I build stuff, I don't mind things being dirty: I can cleanup afterwards. This is especially critical when you don't actually know how to make things clean in the first place, as Haskell will simply not let you do that at all. And obviously, I fought with Monads, or, more specifically, "I/O" or IO in this case. Turns out that getting the current time is IO in Haskell: indeed, it's not a "pure" function that will always return the same thing. But this means that I would have had to change the signature of all the functions that touched time to include IO. I eventually moved the time initialization up into main so that I had only one IO function and moved that timestamp downwards as simple argument. That way I could keep the rest of the code clean, which seems to be an acceptable pattern. I would of course be happy to get feedback from my Haskell readers (if any) to see how to improve that code. I am always eager to learn.

Git remote MediaWiki Few people know that there is a MediaWiki remote for Git which allow you to mirror a MediaWiki site as a Git repository. As a disaster recovery mechanism, I have been keeping such a historical backup of the Amateur radio wiki for a while now. This originally started as a homegrown Python script to also convert the contents in Markdown. My theory then was to see if we could switch from Mediawiki to Ikiwiki, but it took so long to implement that I never completed the work. When someone had the weird idea of renaming a page to some impossible long name on the wiki, my script broke. I tried to look at fixing it and then remember I also had a mirror running using the Git remote. It turns out it also broke on the same issue and that got me looking in the remote again. I got lost in a zillion issues, including fixing that specific issue, but I especially looked at the possibility of fetching all namespaces because I realized that the remote fetches only a part of the wiki by default. And that drove me to submit namespace support as a patch to the git mailing list. Finally, the discussion came back to how to actually maintain that contrib: in git core or outside? Finally, it looks like I'll be doing some maintenance that project outside of git, as I was granted access to the GitHub organisation...

Galore Yak Shaving Then there's the usual hodgepodge of fixes and random things I did over the month.
There is no [web extension] only XUL! - Inside joke

16 October 2017

Iain R. Learmonth: No more no surprises

Debian has generally always had, as a rule, sane defaults and no surprises . This was completely shattered for me when Vim decided to hijack the mouse from my terminal and break all copy/paste functionality. This has occured since the release of Debian 9. I expect for my terminal to behave consistently, and this is broken every time I log in to a Debian 9 system where I have not configured Vim to disable this functionality. I also see I m not alone in this frustration. To fix this, in your .vimrc:
if !has("gui_running")
  set mouse=
endif
(This will check to see if your using GVim or similar, where it would be reasonable to expect the mouse to work.) This is perhaps not aggresive enough though. I never want to have console applications trying to use the mouse. I ve configured rxvt to do things like open URLs in Firefox, etc. that I always want to work, and I always want my local clipboard to be used so I can copy/paste between remote machines. I ve found a small patch that would appear to disable mouse reporting for rxvt, but unfortunately I cannot do this through an Xresources option. If someone is looking for something to do for Hacktoberfest, I d love to see this be an option for rxvt without re-compiling:
diff --git a/src/rxvt.h b/src/rxvt.h
index 5c7cf66..2751ba3 100644
--- a/src/rxvt.h
+++ b/src/rxvt.h
@@ -646,7 +646,7 @@ enum  
 #define PrivMode_ExtMouseRight  (1UL<<24) // xterm pseudo-utf-8, but works in non-utf-8-locales
 #define PrivMode_BlinkingCursor (1UL<<25)
 
-#define PrivMode_mouse_report   (PrivMode_MouseX10 PrivMode_MouseX11 PrivMode_MouseBtnEvent PrivMode_MouseAnyEvent)
+#define PrivMode_mouse_report   0 /* (PrivMode_MouseX10 PrivMode_MouseX11 PrivMode_MouseBtnEvent PrivMode_MouseAnyEvent) */
 
 #ifdef ALLOW_132_MODE
 # define PrivMode_Default (PrivMode_Autowrap PrivMode_ShiftKeys PrivMode_VisibleCursor PrivMode_132OK)

06 August 2017

Joachim Breitner: Communication Failure

I am still far from being a professor, but I recently got a glimps of what awaits you in that role
From: Sebastian R. < @gmail.com>
To: joachim@cis.upenn.edu
Subject: re: Errors I've spotted a basic error in your course on Haskell (https://www.seas.upenn.edu/~cis194/fall16/). Before I proceed, it's cool if you're not receptive to errors being indicated; I've come across a number of professors who would rather take offense than admit we're all human and thus capable of making mistakes... My goal is to find a resource that might be useful well into the future, and a good indicator of that is how responsive the author is to change. In your introduction note you have written:
n contrast to a classical intro into Haskell, we do not start with numbers, booleans, tuples, lists and strings, but we start with pictures. These are of course library-defined (hence the input CodeWorld) and not part of the language . But that does not make them less interesting, and in fact, even the basic boolean type is library defined it just happens to be the standard library.
Howeverm there is no input CodeWorld in the code above. Have you been made aware of this error earlier? Regards, ...
Nice. I like when people learn from my lectures. The introduction is a bit werid, but ok, maybe this guy had some bad experiences. Strangley, I don t see a mistake in the material, so I respond:
From: Joachim Breitner <noscript>joachim at cis dot upenn dot edu</noscript>
To: Sebastian R. < @gmail.com>
Subject: Re: Errors Dear Sebastian, thanks for pointing out errors. But the first piece of code under Basic Haskell starts with
 -# LANGUAGE OverloadedStrings #- 
import CodeWorld
so I am not sure what you are referring to. Note that these are lecture notes, so you have to imagine a lecturer editing code live on stage along with it. If you only have the notes, you might have to infer a few things. Regards, Joachim
A while later, I receive this response:
From: Sebastian R. < @gmail.com>
To: Joachim Breitner <noscript>joachim at cis dot upenn dot edu</noscript>
Subject: Re: Errors Greetings, Joachim. Kindly open the lecture slides and search for "input CodeWorld" to find the error; it is not in the code, but in the paragraph that implicitly refers back to the code. You might note that I quoted this precisely from the lectures... and so I repeat myself... this came from your lectures; they're not my words!
In contrast to a classical intro into Haskell, we do not start with numbers, booleans, tuples, lists and strings, but we start with pictures. These are of course library-defined (hence the input CodeWorld) and not part of the language . But that does not make them less interesting, and in fact, even the basic boolean type is library defined it just happens to be the standard library.
This time around, I've highlighted the issue. I hope that made it easier for you to spot... Nonetheless, I got my answer. Don't reply if you're going to fight tooth and nail about such a basic fix; it's simply a waste of both of our time. I'd rather learn from somewhere else... On Tue, Aug 1, 2017 at 11:19 PM, Joachim Breitner <noscript>joachim at cis dot upenn dot edu</noscript> wrote:
I am a bit reminded of Sean Spicer they re not my words! but clearly I am missing something. And indeed I am: In the code snippet, I wrote correctly import CodeWorld, but in the text I had input CodeWorld. I probably did write LaTeX before writing the lecture notes. Well, glad to have that sorted out. I fixed the mistake and wrote back:
From: Joachim Breitner <noscript>joachim at cis dot upenn dot edu</noscript>
To: Sebastian R. < @gmail.com>
Betreff: Re: Errors Dear Sebastian, nobody is fighting, and I see the mistake now: The problem is not that the line is not in the code, the problem is that there is a typo in the line and I wrote input instead of import . Thanks for the report, although you did turn it into quite a riddle a simple you wrote import when it should have been import would have been a better user of both our time. Regards, Joachim Am Donnerstag, den 03.08.2017, 13:32 +1000 schrieb Sebastian R.:
(And it seems I now made the inverse typo, writing import instead of input . Anyways, I did not think of this any more until a few days later, when I found this nice message in my mailbox:
From: Sebastian R. < @gmail.com>
To: Joachim Breitner <noscript>joachim at cis dot upenn dot edu</noscript>
Subject: Re: Errors
a simple you wrote import when it should have been import would have been a better user of both our time.
We're both programmers. How about I cut ALL of the unnecessary garbage and just tell you to s/import/input/ on that last quotation (the thing immediately before this paragraph, in case you didn't know). I blatantly quoted the error, like this:
In your introduction note you have written:
n contrast to a classical intro into Haskell, we do not start with numbers, booleans, tuples, lists and strings, but we start with pictures. These are of course library-defined (hence the input CodeWorld) and not part of the language . But that does not make them less interesting, and in fact, even the basic boolean type is library defined it just happens to be the standard library.
Howeverm there is no input CodeWorld in the code above.
Since that apparently wasn't clear enough, in my second email to you I had to highlight it like so:
You might note that I quoted this precisely from the lectures... and so I repeat myself... this came from your lectures; they're not my words!
In contrast to a classical intro into Haskell, we do not start with numbers, booleans, tuples, lists and strings, but we start with pictures. These are of course library-defined (hence the input CodeWorld) and not part of the language . But that does not make them less interesting, and in fact, even the basic boolean type is library defined it just happens to be the standard library.
This time around, I've highlighted the issue. I hope that made it easier for you to spot...
I'm not sure if you're memeing at me or not now, but it seems either your reading comprehension, or your logical deduction skills might be substandard. Unfortunately, there isn't much either of us can do about that, so I'm happy to accept that some people will be so stupid; after all, it's to be expected and if we don't accept that which is to be expected then we live our lives in denial. Happy to wrap up this discusson here, Seb... On Fri, Aug 4, 2017 at 12:22 AM, Joachim Breitner <noscript>joachim at cis dot upenn dot edu</noscript> wrote:
Well, I chose to be amused by this, and I am sharing my amusement with you.

04 July 2017

John Goerzen: Time, Frozen

We re expecting a baby any time now. The last few days have had an odd quality of expectation: any time, our family will grow. It makes time seem to freeze, to stand still. We have Jacob, about to start fifth grade and middle school. But here he is, still a sweet and affectionate kid as ever. He loves to care for cats and seeks them out often. He still keeps an eye out for the stuffed butterfly he s had since he was an infant, and will sometimes carry it and a favorite blanket around the house. He will also many days prepare the Yellow House News on his computer, with headlines about his day and some comics pasted in before disappearing to play with Legos for awhile. And Oliver, who will walk up to Laura and give baby a hug many times throughout the day and sneak up to me, try to touch my arm, and say doink before running off before I can doink him back. It was Oliver that had asked for a baby sister for Christmas before he knew he d be getting one! In the past week, we ve had out the garden hose a couple of times. Both boys will enjoy sending mud down our slide, or getting out the water slide to play with, or just playing in mud. The rings of dirt in the bathtub testify to the fun that they had. One evening, I built a fire, we made brats and hot dogs, and then Laura and I sat visiting and watching their water antics for an hour after, laughter and cackles of delight filling the air, and cats resting on our laps. These moments, or countless others like Oliver s baseball games, flying the boys to a festival in Winfield, or their cuddles at bedtime, warm the heart. I remember their younger days too, with fond memories of taking them camping or building a computer with them. Sometimes a part of me wants to just keep soaking in things just as they are; being a parent means both taking pride in children s accomplishments as they grow up, and sometimes also missing the quiet little voice that can be immensely excited by a caterpillar. And yet, all four of us are so excited and eager to welcome a new life into our home. We are ready. I can t wait to hold the baby, or to lay her to sleep, to see her loving and excited older brothers. We hope for a smooth birth, for mom and baby. Here is the crib, ready, complete with a mobile with a cute bear (and even a plane). I can t wait until there is a little person here to enjoy it.

14 May 2017

Bits from Debian: New Debian Developers and Maintainers (March and April 2017)

The following contributors got their Debian Developer accounts in the last two months: The following contributors were added as Debian Maintainers in the last two months: Congratulations!

27 April 2017

Steinar H. Gunderson: Chinese HDMI-to-SDI converters, part II

Following up on my previous post, I have only a few small nuggets of extra information: The power draw appears to be 230 240 mA at 5 V, or about 1.15 W. (It doesn't matter whether they have a signal or not.) This means you can power them off of a regular USB power bank; in a recent test, we used Deltaco 20000 mAh (at 3.7 V) power banks, which are supposed to power a GoPro plus such a converter for somewhere between 8 12 hours (we haven't measured exactly). It worked brilliantly, and solved a headache of how to get AC to the camera and converter; just slap on a power bank instead, and all you need to run is SDI. Curiously enough, 230 mA is so little that the power bank thinks it doesn't count as load, and just turns itself off after ten seconds or so. However, with the GoPro also active, it stays on all the time. At least it ran for the two hours that we needed without a hitch. The SDI converters don't accept USB directly, but you can purchase dumb USB-to-5.5x2.1mm converters cheap from wherever, which works fine even though USB is supposed to give you only 100 mA without handshaking. Some eBay sellers seem to include them with the converters, even. I guess the power bank just doesn't care; it's spec-ed to 2.1 A on two ports and 1 A on the last one. Update: Sebastian Reichel pointed out that USB 2.0 ups the limit to 500 mA, so you should be within spec in any case. :-) And here's a picture of the entire contraption as a bonus: Power bank and HDMI-to-SDI converter

05 February 2017

Iustin Pop: Short rant/review of La La Land

Warning: Spoilers below. Rant below. Much angry, MANY ALL-CAPS. You've been warned! So, today we went to see "La La Land", because I've heard good things about it, and because I do enjoy good musicals. And because of this, I wrote this post, instead of what I originally had in mind (related to kernel configuration). Was it a good movie? Definitely yes. Was it a good musical? So and so. Did I like the ending? HELL NO, over and over NO. The movie itself was much better than I expected. I don't read plot details in advance nor real reviews, so I expected more of a musical, and less of a good plot. But the movie had a very good plot. Two young people, striving to fulfil their artistic dreams, fall in love, and they fight through-sometimes helping, sometimes hindering each other until, finally, each gets their own breakthrough, etc. The choice of actress was spot on halfway through the movie, I was thinking that I can't imagine the same plot played by a different actress. Of course many other actresses could have played the part, but Emma Stone played so well, I have trouble seeing the same character with the same always half-happy, half-sad attitude. The choice of actor was I think OK at first I was in doubt, but he played also well. Or maybe it was just that I couldn't identify with him at first. Not that I identify well with artists in general The dance scenes were OK, and the singing good, but as I said, the musical part was secondary to the actual struggles of the characters. The movie itself was, technically, very well done; a lot of filming was in bars/clubs/locations with difficult lighting, and the shooting was very good. They also had a scene on a pier, looking towards the ocean and the setting sun, and the characters walking towards the beach so heavily back-lighted, and I kept thinking "If I get only one shot this perfectly exposed and colour correct(ed), I'm happy". So high notes here. Back to the plot. The story of how she and him fought their own struggles was very nicely told. Tick-tack, up and down (hope and rejection), leaning on the other to get morale back, is a captivating story. The cliff-hanger at the pre-end with her career, the going back home, the last minute save, all very well told. So at this stage, I would have given the movie a 9/10. And I was happy. Then we have the usual "one character has to go away to a far away country for a long time", except in this case it was just 4 months. And they have the usual discussion "what do we do with our relation, where do we take it", and she says "I will always love you", to which he replies "And I will too" (or equivalent). In my mind, this means they'll have to survive during the break, they'll have to also survive through his touring months/years, but in the end love will be stronger. Because this is what the movie told us until now, that she made it because of him, and he made it because of her. Neither of them would have been this strong without the other (he wouldn't have picked up the invitation from his old pal, she wouldn't have gone to the final audition request nor write the play which got her the audition/recognition). Estimated movie ending: awesome. And then something happens. The timeline jumps 5 years in the future (as expected), and she is famous, married (WITH SOMEONE ELSE) and happy mother of a 3-year old. Through fate, she and her husband enter the club of Sebastian (as he also fulfilled his dream), she and Sebastian see each other, he plays their song, during which we're served a re-run of the movie but in stupid "everything goes well" style (all bad events eliminated), in which it is she and Sebastian who enter the club (which belongs now to somebody else), and then we're back in real time, song ends, she and her husband leave, but before that she and Sebastian exchange one last smile, THE END. And I'm sitting there, not believing my eyes. WHAT THE? So I get home, not write this post for four hours to calm down, but I can't. Because this doesn't make sense. AT ALL. What does the internet say? Quoting from this CNN article, written exactly today. The director says:
"That ending was there from the get-go," [director Damien Chazelle] told CNN in a recent interview. "I think I just have a thing about love stories where the lovers don't wind up together at the end; I find it very romantic."
Huh, excuse me?
"I think there's a reason why most of the greatest love stories in history don't end with happily ever after," Chazelle said. "To me, if you're telling a story about love, love has to be bigger than the characters." Chazelle sees Mia and Sebastian's love as a "third character" and something that "lives on." "[The ending gives] you that sense that even if the relationship itself might be over in practical terms, the love is not over," he said. "The love lasts, and I think that's just a beautiful kind of thing."
OH FOR THE LOVE OF. This is a wishy-washy explanation that tries to approach the thing from the artistic side. No, this is bullshit, because of multiple things. Let me try to roll back and explain what I think was the intention.
  1. An earlier fight between Mia and Sebastian points to the fact that they're both very dedicated to their careers, and this means it's hard for them to stay together if they both chase their dream. He has to be on tour, and she has to rehearse for her play, so they won't see each other for at least two weeks (in this instance). Later, she calls him and leaves a message that she hasn't seen him in a while (complex scene which ends in another fight, which is very well done). So we see the conflict that seems to say "You can't have a relation of equals; one party has to give up their dream". While this might be partially true in the real world, I don't go to movies to see the real world.
  2. After the year-long window into their life, I can't think that either Sebastian or Mia can be really successful without the other; because they are so alike, so passionate about their dreams, that a normal person wouldn't be able to understand and push the other when they need. However, the ending show both Mia and Sebastian quite successful, so one has to wonder: did they make it alone? Sebastian seems so (we don't see a partner for him), Mia unclear, likely not. How did Sebastian get through? What did Mia find in her husband?
  3. This is very one-sided, since I'm a man, so bear with me: Sebastian helped Mia through her tough time. Once she got the breakthrough (and they split), she found somebody else, and I have to wonder in what circumstances they met. In the sense that maybe her husband only knew "successful Mia" and not "struggling/aspiring Mia". Her husband seems completely oblivious to all the eye contact between Mia and Sebastian in the club, seems to know Sebastian/about Sebastian not. How deep is their relation?
  4. This is still one sided, sorry. When they break up (before Mia leaves for Paris), Sebastian asks "so where do we go from here?". Mia says "Nowhere". He asks once more, she rejects him again. So after one year of mad love and cries and happy moments, he gives up over two sentences? He's been following his dream (proper Jazz) in spite of all downturns in life until then, but he gives up on his real love over this? It doesn't make sense; trying to identify my self with the character, I can't reconcile this scene at all, unless he didn't really love her.
So no, I don't see them ending apart as romantic. I see it as the director is saying "You can't have both love and your [career] dreams. Choose either.", and he gives the "love" fake ending in the mini-re-roll of the movie, and the "career" wrong ending in the actual ending. And worse, he does it by negating significant parts of the character development done until now. Moreover, this conclusion is wrong. Wrong because this is a movie, and if movies don't manage to make you dream that you can achieve all, if movies tell you "choose either", then all is lost. Their love is not a separate character; them struggling to find each other in the successful phase of their life, learning to adapt to the new "he" and "she", would be the third thing. As it was shown, their love is simply a young love, that can't really survive the changes in life; they each said "I'll love you forever", but with this ending it sounds more "I'll cherish the memory of young you forever". Or differently said, it sounded like a cheap excuse to use when ending their relationship, in order to not negate the relationship itself. My version of the movie is another half hour long. It explains how Sebastian get over the "only jazz is pure old jazz" and manages to build a successful business around his old-style-but-modern jazz, instead of the pop-style jazz of the touring band (while thinking about her). It explains how Mia becomes a successful actress and gets over her first/second movies (while thinking about him), because one movie doesn't make one really successful (that reminds me: 3 year old child after 5 year forward-jump? when/how did her career go?). Hell, make it even more bitter show how their correspondence starts strong but becomes more and more sporadic over time, dying after the first 2 years. Show how both of them try other relations, and not find the same spark that they had before. And then, after they have matured, they meet again. And, just like the first time, they fall for each other, once again. She for his music, him for her passion for acting/for acting itself. She finds that him naming his club after her suggestion is oh-so-grown-up-and-sweet, he is happy that she finally grew into what he saw in her from the beginning. And he sings their song once more. But no. I'm not an artist, so I can only get the "die hope die die die love because I can" version. I still recommend the movie, but not the "after 5 years" scenes. Also, I didn't get time to bike today nor yesterday, so all you really get here is an ANGRY RANT. Because while I drink the coffee black and the tea without sugar, I like my happy endings, DAMN IT.

28 January 2017

Bits from Debian: Debian at FOSDEM 2017

On February 4th and 5th, Debian will be attending FOSDEM 2017 in Brussels, Belgium; a yearly gratis event (no registration needed) run by volunteers from the Open Source and Free Software community. It's free, and it's big: more than 600 speakers, over 600 events, in 29 rooms. This year more than 45 current or past Debian contributors will speak at FOSDEM: Alexandre Viau, Bradley M. Kuhn, Daniel Pocock, Guus Sliepen, Johan Van de Wauw, John Sullivan, Josh Triplett, Julien Danjou, Keith Packard, Martin Pitt, Peter Van Eynde, Richard Hartmann, Sebastian Dr ge, Stefano Zacchiroli and Wouter Verhelst, among others. Similar to previous years, the event will be hosted at Universit libre de Bruxelles. Debian contributors and enthusiasts will be taking shifts at the Debian stand with gadgets, T-Shirts and swag. You can find us at stand number 4 in building K, 1 B; CoreOS Linux and PostgreSQL will be our neighbours. See https://wiki.debian.org/DebianEvents/be/2017/FOSDEM for more details. We are looking forward to meeting you all!

08 January 2017

Bits from Debian: New Debian Developers and Maintainers (November and December 2016)

The following contributors got their Debian Developer accounts in the last two months: The following contributors were added as Debian Maintainers in the last two months: Congratulations!

25 December 2016

Gregor Herrmann: RC bugs 2016/46-51

it's amazing how many new bugs appear; luckily in the Debian Perl Group we're not too bad at fixing them as well. here's the list of my contributions over the last weeks:

07 December 2016

Shirish Agarwal: Day trip in Cape Town, part 2

Debconf16 logo The post continues from the last post shared. Let me get some interesting tit-bits not related to the day-trip out-of-the-way first I don t know whether we had full access to see all parts of fuller hall or not. Couple of days I was wondering around Fuller Hall, specifically next to where clothes were pressed. Came to know of the laundry service pretty late but still was useful. Umm next to where the ladies/gentleman pressed our clothes, there is a stairway which goes down. In fact even on the opposite side there is a stairway which goes down. I dunno if other people explored them or not. The jail inside and under UCT I was surprised and shocked to see bars in each room as well as connecting walkways etc. I felt a bit sad, confused and curious and went on to find more places like that. After a while I came up to the ground-level and enquired with some of the ladies therein. I was shocked to know that UCT some years ago (they were not specific) was a jail for people. I couldn t imagine that a place which has so much warmth (in people, not climate) could be evil in a sense. I was not able to get much information out of them about the nature of jail it was, maybe it is a dark past that nobody wants to open up, dunno. There were also two *important* aspects of UCT which Bernelle either forgot, didn t share or I just came to know via the Wikipedia page then but nothing else. 1. MeerKAT Apparently quite a bit of the technology was built-in UCT itself. This would have been interesting for geeks and wanna-be geeks like me 2. The OpenContent Initiative by UCT This would have been also something worth exploring. One more interesting thing which I saw was the French council in Cape Town from outside The French Council in cape town from outside I would urge to look at the picture in the gallery as the picture I shared doesn t really show all the details. For e.g. the typical large french windows which are the hall-mark of French architecture doesn t show its glory but if you look at 1306 2322 original picture instead of the 202 360 reproduction you will see that. You will also the insignia of the French Imperial Eagle whose history I came to know only after I looked it up on the Wikipedia page on that day. It seemed fascinating and probably would have the same pride as the State Emblem of India has for Indians with the four Asiatic Lions standing in a circle protecting each other. I also like the palm tree and the way the French Council seemed little and yet had character around all the big buildings. What also was interesting that there wasn t any scare/fear-build and we could take photos from outside unlike what I had seen and experienced in Doha, Qatar as far as photography near Western Embassies/Councils were concerned. One of the very eye-opening moments for me was also while I was researching flights from India to South Africa. While perhaps unconsciously I might have known that Middle East is close to India, in reality, it was only during the search I became aware that most places in Middle East by flight are only an hour or two away. This was shocking as there is virtually no mention of one of our neighbours when they are source of large-scale remittances every year. I mean this should have been in our history and geography books but most do not dwell on the subject. It was only during and after that I could understand Mr. Modi s interactions and trade policies with the Middle East. Another interesting bit was seeing a bar in a Sprinbok bus spingbok atlas bar in bus While admittedly it is not the best picture of the bar, I was surprised to find a bar at the back of a bus. By bar I mean a machine which can serve anything from juices to alcoholic drinks depending upon what is stocked. What was also interesting in the same bus is that the bus also had a middle entrance-and-exit. The middle door in springbok atlas This is something I hadn t seen in most Indian buses. Some of the Volvo buses have but it is rarely used (only except emergencies) . An exhaustive showcase of local buses can be seen here . I find the hand-drawn/cad depictions of all the buses by Amit Pense near to the T. Axe which can be used to break windows Emergency exit window This is also something which I have not observed in Indian inter-city buses (axe to break the window in case of accident and breakable glass which doesn t hurt anyone I presume), whether they are State-Transport or the high-end Volvo s . Either it s part of South African Roads Regulations or something that Springbok buses do for their customers. All of these queries about the different facets I wanted to ask the bus-driver and the attendant/controller but in the excitement of seeing, recording new things couldn t ask In fact one of the more interesting things I looked at and could look day and night is the variety of vehicles on display in Cape Town. In hindsight, I should have bought a couple of 128 GB MMC cards for my mobile rather than the 64 GB one. It was just plain inadequate to capture all that was new and interesting. Auditorum chair truck seen near Auditorium This truck I had seen about some 100 metres near the Auditorium on Upper Campus. The truck s design, paint was something I had never seen before. It is/was similar to casket trucks seen in movies but the way it was painted and everything made it special. What was interesting is to see the gamut of different vehicles. For instance, there were no bicycles that I saw in most places. There were mostly Japanese/Italian bikes and all sorts of trucks. If I had known before, I would definitely have bought an SD specifically to take snaps of all the different types of trucks, cars etc. that I saw therein. The adage/phrase I should stop in any one place and the whole world will pass me by seemed true on quite a few South African Roads. While the roads were on par or a shade better than India, many of those were wide roads. Seeing those, I was left imagining how the Autobahn in Germany and other high-speed Expressways would look n feel. India has also been doing that with the Pune-Mumbai Expressway and projects like Yamuna Expressway and now the extension Agra Lucknow Expressway but doing this all over India would take probably a decade or more. We have been doing it since a decade and a half. NHDP and PMGSY are two projects which are still ongoing to better the roads. We have been having issues as to should we have toll or no toll issues but that is a discussion for some other time. One of the more interesting sights I saw was the high-arched gothic-styled church from outside. This is near Longstreet as well. high arch gothic-styled church I have seen something similar in Goa, Pondicherry but not such high-arches. I did try couple of times to gain entry but one time it was closed, the other time some repairing/construction work was going on or something. I would loved to see it from inside and hopefully they would have had an organ (music) as well. I could imagine to some extent the sort of music that would have come out. Now that Goa has come in the conversation I can t help but state that Seafood enthusiasts/lover/aficionado, or/and Pescatarianism would have a ball of a time in Goa. Goa is on the Konkan coast and while I m eggie, ones who enjoy seafood really have a ball of a time in Goa. Fouthama s Festival which happens in February is particularly attractive as Goan homes are thrown open for people to come and sample their food, exchange recipes and alike. This happens around 2 weeks before the Goan Carnival and is very much a part of the mish-mashed Konkani-Bengali-Parsi-Portugese culture. I better stop here about the Goa otherwise I ll get into reminiscing mode. To put the story and event back on track from where we left of (no fiction hereon), Nicholas was in constant communication with base, i.e. UCT as well as another group who was hiking from UCT to Table Mountain. We waited for the other group to join us till 13:00 hrs. We came to know that they were lost and were trying to come up and hence would take more time. As Bernelle was with them, who was a local and she had two dogs who knew the hills quite well, it was decided to go ahead without them. We came down the same cable-car and then ventured on towards Houtbay. Houtbay has it all, a fisherman s wharf, actual boats with tough-mean looking men with tattoos working on boats puffing cigars/pipes, gaggle of sea-gulls, the whole scene. Sharing a few pictures of the way in-between. the view en-route to Houtbay western style car paint and repair shop Tajmahal Indian Restaurant, Houtbay I just now had a quick look at the restaurant and it seems they had options for veggies too. Unfortunately, the rating leaves a bit to be desired but then dunno as Indian flavoring is something that takes time to get used too. Zomato doesn t give any idea of from when a restaurant is in business and has too few reviews so not easy to know how the experience would have been. Chinese noodles and small houses Notice the pattern, the pattern of small houses I saw all the way till Houtbay and back. I do vaguely remember starting a discussion about it on the bus but don t really remember. I have seen (on TV) cities like Miami, Dubai or/and Hong Kong who have big buildings on the beach but both in Konkan as well as Houtbay there were small buildings. I guess a combination of zoning regulations, feel of community, fear of being flooded all play into beaches being the way they are. Also, this probably is good as less stress on the environment. Miamiboyz from Wikimedia Commons The above picture is taken from Wikipedia from the article Miami Beach, Florida for comparison. Audi rare car to be seen in India The Audi rare car to be seen in India. This car has been associated with Ravi Shastri when he won it in 1985. I was young but still get goosebumps remembering those days. first-glance-Houtbay-and-pier First glance of Houtbay beach and pier. Notice how clean and white the beach is. Wharf-Grill-Restaurant-from-side-and-Hop-on-Hop-off-bus You can see the wharf grill restaurant in the distance (side-view), see the back of the hop on and hop off bus (a concept which was unknown to me till then). Once I came back and explored on the web came to know this concept is prevalent in many a touristy places around the world. Umm also By sheer happenchance also captured a beautiful looking Indian female . So many things happening all at once In Hindi, we would call this picture virodabhas or contradiction . this is in afternoon, around 1430 hrs. You have the sun, the clouds, the Mountains, the x number of boats, the pier, the houses, the cars, the shops. It was all crazy and beautiful at the same time. The Biggest Contradiction is seeing the Mountain, the beach and the Sea in the same Picture. Baffled the mind. Konkan though is a bit similar there as well. You have all the three things in some places but that s a different experience altogether as ours is a more tropical weather although is one of the most romantic places in the rains. We were supposed to go on a short cruise to seal/dolphin island but as we were late (as had been waiting for the other group) didn t go and instead just loitered there. Fake-real lookout bar-restaurant IIRC the lookout bar is situated just next to Houtbay Search and Rescue. Although was curious if the Lookout tower was used in case of disappearance. lost people, boats etc. Seal in action Seal jumping over water, what a miracle ! One of the boats on which we possibly could have been on. It looked like the boat we could have been on. I clicked as I especially liked the name Calypso and Calypso . I shared the two links as the mythologies, interpretation differ a bit between Greek and Hollywood culture Debian folks and the area around Can see few Debian folks in the foreground, next to the Pole and the area around. Also can see a bit of the area around. Alone boy trying to surf I don t know anything about water sports and after sometime he came out. I was left wondering though, how safe he was in that water. While he was close to the pier and he was just paddling, there weren t big waves still felt a bit of concern. Mr. Seal - the actor and his handler While the act was not to the level we see in the movies, still for the time I hung around, I saw him showing attitude for his younger audiences, eating out of their hands, making funny sounds. Btw he farted a few times, whether that was a put-on or not can t really say but produced a few guffaws from his audience. A family feeding Mr. Seal I dunno why the birds came down for. Mr. Seal was being fed oily small fish parts, dunno if the oil was secreted by the fish themselves or whatever, it just looked oily from distance. Bird-Man-Bird Bird taking necessary sun bath typical equipment on a boat to catch fish-lot of nets boats-nets-and-ropes People working on disentangling a net There wasn t much activity on the time we went. It probably would have been different on sunrise and would be on sunset. The only activity I saw was on this boat where they were busy fixing and disentangling the lines. I came up with 5-15 different ideas for a story but rejected them as a. Probably all of them have been tried. People have been fishing since the beginning of time and modern fishing probably 200 odd years or so. I have read accounts of fishing companies in early 1800s onwards, so probably all must have been tried. b. More dangerous one, if there is a unique idea, then it becomes more dangerous as writing is an all-consuming process. Writing a blog post (bad or good) takes lots of time. I constantly read, re-read, try and improvise till I can or my patience loses out. In book you simply can t have such luxuries. hout-bay-search-and-rescue-no-parking-zone No parking/tow zone in/near the Houtbay search and rescue. Probably to take out emergency vehicles once something untoward happens. hout-bay-sea-rescue-with-stats Saved 54 lives, boats towed 154 Salut! Houtbay sea rescue. The different springbok atlas bus that we were on kraal-kraft The only small criticism is for Houtbay there wasn t a single public toilet. We had to ask favor at kraal kraft to use their toilets and there could have been accidents, it wasn t lighted well and water was spilled around. Road sign telling that we are near to UCT For us, because we were late we missed both the boat-cruise as well as some street shops selling trinkets. Other than that it was all well. We should have stayed till sunset, I am sure the view would have been breath-taking but we hadn t booked the bus till evening. Back at UCT Overall it was an interesting day as we had explored part of Table Mountain, seen the somewhat outrageously priced trinkets there as well as explored Houtbay sea-side as well.
Filed under: Miscellenous Tagged: #Audi, #Cape Town, #Cruises, #Debconf16, #French Council, #Geography, #Houtbay Sea Rescue, #Jail, #Middle East, #Springbok Atlas, #Vehicles

26 October 2016

Joachim Breitner: Showcasing Applicative

My plan for this week s lecture of the CIS 194 Haskell course at the University of Pennsylvania is to dwell a bit on the concept of Functor, Applicative and Monad, and to highlight the value of the Applicative abstraction. I quite like the example that I came up with, so I want to share it here. In the interest of long-term archival and stand-alone presentation, I include all the material in this post.1

Imports In case you want to follow along, start with these imports:
import Data.Char
import Data.Maybe
import Data.List
import System.Environment
import System.IO
import System.Exit

The parser The starting point for this exercise is a fairly standard parser-combinator monad, which happens to be the result of the student s homework from last week:
newtype Parser a = P (String -> Maybe (a, String))
runParser :: Parser t -> String -> Maybe (t, String)
runParser (P p) = p
parse :: Parser a -> String -> Maybe a
parse p input = case runParser p input of
    Just (result, "") -> Just result
    _ -> Nothing -- handles both no result and leftover input
noParserP :: Parser a
noParserP = P (\_ -> Nothing)
pureParserP :: a -> Parser a
pureParserP x = P (\input -> Just (x,input))
instance Functor Parser where
    fmap f p = P $ \input -> do
	(x, rest) <- runParser p input
	return (f x, rest)
instance Applicative Parser where
    pure = pureParserP
    p1 <*> p2 = P $ \input -> do
        (f, rest1) <- runParser p1 input
        (x, rest2) <- runParser p2 rest1
        return (f x, rest2)
instance Monad Parser where
    return = pure
    p1 >>= k = P $ \input -> do
        (x, rest1) <- runParser p1 input
        runParser (k x) rest1
anyCharP :: Parser Char
anyCharP = P $ \input -> case input of
    (c:rest) -> Just (c, rest)
    []       -> Nothing
charP :: Char -> Parser ()
charP c = do
    c' <- anyCharP
    if c == c' then return ()
               else noParserP
anyCharButP :: Char -> Parser Char
anyCharButP c = do
    c' <- anyCharP
    if c /= c' then return c'
               else noParserP
letterOrDigitP :: Parser Char
letterOrDigitP = do
    c <- anyCharP
    if isAlphaNum c then return c else noParserP
orElseP :: Parser a -> Parser a -> Parser a
orElseP p1 p2 = P $ \input -> case runParser p1 input of
    Just r -> Just r
    Nothing -> runParser p2 input
manyP :: Parser a -> Parser [a]
manyP p = (pure (:) <*> p <*> manyP p)  orElseP  pure []
many1P :: Parser a -> Parser [a]
many1P p = pure (:) <*> p <*> manyP p
sepByP :: Parser a -> Parser () -> Parser [a]
sepByP p1 p2 = (pure (:) <*> p1 <*> (manyP (p2 *> p1)))  orElseP  pure []
A parser using this library for, for example, CSV files could take this form:
parseCSVP :: Parser [[String]]
parseCSVP = manyP parseLine
  where
    parseLine = parseCell  sepByP  charP ',' <* charP '\n'
    parseCell = do
        charP '"'
        content <- manyP (anyCharButP '"')
        charP '"'
        return content

We want EBNF Often when we write a parser for a file format, we might also want to have a formal specification of the format. A common form for such a specification is EBNF. This might look as follows, for a CSV file:
cell = '"',  not-quote , '"';
line = (cell,  ',', cell    ''), newline;
csv  =  line ;
It is straightforward to create a Haskell data type to represent an EBNF syntax description. Here is a simple EBNF library (data type and pretty-printer) for your convenience:
data RHS
  = Terminal String
    NonTerminal String
    Choice RHS RHS
    Sequence RHS RHS
    Optional RHS
    Repetition RHS
  deriving (Show, Eq)
ppRHS :: RHS -> String
ppRHS = go 0
  where
    go _ (Terminal s)     = surround "'" "'" $ concatMap quote s
    go _ (NonTerminal s)  = s
    go a (Choice x1 x2)   = p a 1 $ go 1 x1 ++ "   " ++ go 1 x2
    go a (Sequence x1 x2) = p a 2 $ go 2 x1 ++ ", "  ++ go 2 x2
    go _ (Optional x)     = surround "[" "]" $ go 0 x
    go _ (Repetition x)   = surround " " " " $ go 0 x
    surround c1 c2 x = c1 ++ x ++ c2
    p a n   a > n     = surround "(" ")"
            otherwise = id
    quote '\'' = "\\'"
    quote '\\' = "\\\\"
    quote c    = [c]
type Production = (String, RHS)
type BNF = [Production]
ppBNF :: BNF -> String
ppBNF = unlines . map (\(i,rhs) -> i ++ " = " ++ ppRHS rhs ++ ";")

Code to produce EBNF We had a good time writing combinators that create complex parsers from primitive pieces. Let us do the same for EBNF grammars. We could simply work on the RHS type directly, but we can do something more nifty: We create a data type that keeps track, via a phantom type parameter, of what Haskell type the given EBNF syntax is the specification:
newtype Grammar a = G RHS
ppGrammar :: Grammar a -> String
ppGrammar (G rhs) = ppRHS rhs
So a value of type Grammar t is a description of the textual representation of the Haskell type t. Here is one simple example:
anyCharG :: Grammar Char
anyCharG = G (NonTerminal "char")
Here is another one. This one does not describe any interesting Haskell type, but is useful when spelling out the special characters in the syntax described by the grammar:
charG :: Char -> Grammar ()
charG c = G (Terminal [c])
A combinator that creates new grammar from two existing grammars:
orElseG :: Grammar a -> Grammar a -> Grammar a
orElseG (G rhs1) (G rhs2) = G (Choice rhs1 rhs2)
We want the convenience of our well-known type classes in order to combine these values some more:
instance Functor Grammar where
    fmap _ (G rhs) = G rhs
instance Applicative Grammar where
    pure x = G (Terminal "")
    (G rhs1) <*> (G rhs2) = G (Sequence rhs1 rhs2)
Note how the Functor instance does not actually use the function. How should it? There are no values inside a Grammar! We cannot define a Monad instance for Grammar: We would start with (G rhs1) >>= k = , but there is simply no way of getting a value of type a that we can feed to k. So we will do without a Monad instance. This is interesting, and we will come back to that later. Like with the parser, we can now begin to build on the primitive example to build more complicated combinators:
manyG :: Grammar a -> Grammar [a]
manyG p = (pure (:) <*> p <*> manyG p)  orElseG  pure []
many1G :: Grammar a -> Grammar [a]
many1G p = pure (:) <*> p <*> manyG p
sepByG :: Grammar a -> Grammar () -> Grammar [a]
sepByG p1 p2 = ((:) <$> p1 <*> (manyG (p2 *> p1)))  orElseG  pure []
Let us run a small example:
dottedWordsG :: Grammar [String]
dottedWordsG = many1G (manyG anyCharG <* charG '.')
*Main> putStrLn $ ppGrammar dottedWordsG
'', ('', char, ('', char, ('', char, ('', char, ('', char, ('',  
Oh my, that is not good. Looks like the recursion in manyG does not work well, so we need to avoid that. But anyways we want to be explicit in the EBNF grammars about where something can be repeated, so let us just make many a primitive:
manyG :: Grammar a -> Grammar [a]
manyG (G rhs) = G (Repetition rhs)
With this definition, we already get a simple grammar for dottedWordsG:
*Main> putStrLn $ ppGrammar dottedWordsG
'',  char , '.',  char , '.' 
This already looks like a proper EBNF grammar. One thing that is not nice about it is that there is an empty string ('') in a sequence ( , ). We do not want that. Why is it there in the first place? Because our Applicative instance is not lawful! Remember that pure id <*> g == g should hold. One way to achieve that is to improve the Applicative instance to optimize this case away:
instance Applicative Grammar where
    pure x = G (Terminal "")
    G (Terminal "") <*> G rhs2 = G rhs2
    G rhs1 <*> G (Terminal "") = G rhs1
    (G rhs1) <*> (G rhs2) = G (Sequence rhs1 rhs2)
	 
Now we get what we want:
*Main> putStrLn $ ppGrammar dottedWordsG
 char , '.',  char , '.' 
Remember our parser for CSV files above? Let me repeat it here, this time using only Applicative combinators, i.e. avoiding (>>=), (>>), return and do-notation:
parseCSVP :: Parser [[String]]
parseCSVP = manyP parseLine
  where
    parseLine = parseCell  sepByP  charG ',' <* charP '\n'
    parseCell = charP '"' *> manyP (anyCharButP '"') <* charP '"'
And now we try to rewrite the code to produce Grammar instead of Parser. This is straightforward with the exception of anyCharButP. The parser code for that inherently monadic, and we just do not have a monad instance. So we work around the issue by making that a primitive grammar, i.e. introducing a non-terminal in the EBNF without a production rule pretty much like we did for anyCharG:
primitiveG :: String -> Grammar a
primitiveG s = G (NonTerminal s)
parseCSVG :: Grammar [[String]]
parseCSVG = manyG parseLine
  where
    parseLine = parseCell  sepByG  charG ',' <* charG '\n'
    parseCell = charG '"' *> manyG (primitiveG "not-quote") <* charG '"'
Of course the names parse are not quite right any more, but let us just leave that for now. Here is the result:
*Main> putStrLn $ ppGrammar parseCSVG
 ('"',  not-quote , '"',  ',', '"',  not-quote , '"'    ''), '
' 
The line break is weird. We do not really want newlines in the grammar. So let us make that primitive as well, and replace charG '\n' with newlineG:
newlineG :: Grammar ()
newlineG = primitiveG "newline"
Now we get
*Main> putStrLn $ ppGrammar parseCSVG
 ('"',  not-quote , '"',  ',', '"',  not-quote , '"'    ''), newline 
which is nice and correct, but still not quite the easily readable EBNF that we saw further up.

Code to produce EBNF, with productions We currently let our grammars produce only the right-hand side of one EBNF production, but really, we want to produce a RHS that may refer to other productions. So let us change the type accordingly:
newtype Grammar a = G (BNF, RHS)
runGrammer :: String -> Grammar a -> BNF
runGrammer main (G (prods, rhs)) = prods ++ [(main, rhs)]
ppGrammar :: String -> Grammar a -> String
ppGrammar main g = ppBNF $ runGrammer main g
Now we have to adjust all our primitive combinators (but not the derived ones!):
charG :: Char -> Grammar ()
charG c = G ([], Terminal [c])
anyCharG :: Grammar Char
anyCharG = G ([], NonTerminal "char")
manyG :: Grammar a -> Grammar [a]
manyG (G (prods, rhs)) = G (prods, Repetition rhs)
mergeProds :: [Production] -> [Production] -> [Production]
mergeProds prods1 prods2 = nub $ prods1 ++ prods2
orElseG :: Grammar a -> Grammar a -> Grammar a
orElseG (G (prods1, rhs1)) (G (prods2, rhs2))
    = G (mergeProds prods1 prods2, Choice rhs1 rhs2)
instance Functor Grammar where
    fmap _ (G bnf) = G bnf
instance Applicative Grammar where
    pure x = G ([], Terminal "")
    G (prods1, Terminal "") <*> G (prods2, rhs2)
        = G (mergeProds prods1 prods2, rhs2)
    G (prods1, rhs1) <*> G (prods2, Terminal "")
        = G (mergeProds prods1 prods2, rhs1)
    G (prods1, rhs1) <*> G (prods2, rhs2)
        = G (mergeProds prods1 prods2, Sequence rhs1 rhs2)
primitiveG :: String -> Grammar a
primitiveG s = G (NonTerminal s)
The use of nub when combining productions removes duplicates that might be used in different parts of the grammar. Not efficient, but good enough for now. Did we gain anything? Not yet:
*Main> putStr $ ppGrammar "csv" (parseCSVG)
csv =  ('"',  not-quote , '"',  ',', '"',  not-quote , '"'    ''), newline ;
But we can now introduce a function that lets us tell the system where to give names to a piece of grammar:
nonTerminal :: String -> Grammar a -> Grammar a
nonTerminal name (G (prods, rhs))
  = G (prods ++ [(name, rhs)], NonTerminal name)
Ample use of this in parseCSVG yields the desired result:
parseCSVG :: Grammar [[String]]
parseCSVG = manyG parseLine
  where
    parseLine = nonTerminal "line" $
        parseCell  sepByG  charG ',' <* newline
    parseCell = nonTerminal "cell" $
        charG '"' *> manyG (primitiveG "not-quote") <* charG '"
*Main> putStr $ ppGrammar "csv" (parseCSVG)
cell = '"',  not-quote , '"';
line = (cell,  ',', cell    ''), newline;
csv =  line ;
This is great!

Unifying parsing and grammar-generating Note how simliar parseCSVG and parseCSVP are! Would it not be great if we could implement that functionality only once, and get both a parser and a grammar description out of it? This way, the two would never be out of sync! And surely this must be possible. The tool to reach for is of course to define a type class that abstracts over the parts where Parser and Grammer differ. So we have to identify all functions that are primitive in one of the two worlds, and turn them into type class methods. This includes char and orElse. It includes many, too: Although manyP is not primitive, manyG is. It also includes nonTerminal, which does not exist in the world of parsers (yet), but we need it for the grammars. The primitiveG function is tricky. We use it in grammars when the code that we might use while parsing is not expressible as a grammar. So the solution is to let it take two arguments: A String, when used as a descriptive non-terminal in a grammar, and a Parser a, used in the parsing code. Finally, the type classes that we except, Applicative (and thus Functor), are added as constraints on our type class:
class Applicative f => Descr f where
    char :: Char -> f ()
    many :: f a -> f [a]
    orElse :: f a -> f a -> f a
    primitive :: String -> Parser a -> f a
    nonTerminal :: String -> f a -> f a
The instances are easily written:
instance Descr Parser where
    char = charP
    many = manyP
    orElse = orElseP
    primitive _ p = p
    nonTerminal _ p = p
instance Descr Grammar where
    char = charG
    many = manyG
    orElse = orElseG
    primitive s _ = primitiveG s
    nonTerminal s g = nonTerminal s g
And we can now take the derived definitions, of which so far we had two copies, and define them once and for all:
many1 :: Descr f => f a -> f [a]
many1 p = pure (:) <*> p <*> many p
anyChar :: Descr f => f Char
anyChar = primitive "char" anyCharP
dottedWords :: Descr f => f [String]
dottedWords = many1 (many anyChar <* char '.')
sepBy :: Descr f => f a -> f () -> f [a]
sepBy p1 p2 = ((:) <$> p1 <*> (many (p2 *> p1)))  orElse  pure []
newline :: Descr f => f ()
newline = primitive "newline" (charP '\n')
And thus we now have our CSV parser/grammar generator:
parseCSV :: Descr f => f [[String]]
parseCSV = many parseLine
  where
    parseLine = nonTerminal "line" $
        parseCell  sepBy  char ',' <* newline
    parseCell = nonTerminal "cell" $
        char '"' *> many (primitive "not-quote" (anyCharButP '"')) <* char '"'
We can now use this definition both to parse and to generate grammars:
*Main> putStr $ ppGrammar2 "csv" (parseCSV)
cell = '"',  not-quote , '"';
line = (cell,  ',', cell    ''), newline;
csv =  line ;
*Main> parse parseCSV "\"ab\",\"cd\"\n\"\",\"de\"\n\n"
Just [["ab","cd"],["","de"],[]]

The INI file parser and grammar As a final exercise, let us transform the INI file parser into a combined thing. Here is the parser (another artifact of last week s homework) again using applicative style2:
parseINIP :: Parser INIFile
parseINIP = many1P parseSection
  where
    parseSection =
        (,) <$  charP '['
            <*> parseIdent
            <*  charP ']'
            <*  charP '\n'
            <*> (catMaybes <$> manyP parseLine)
    parseIdent = many1P letterOrDigitP
    parseLine = parseDecl  orElseP  parseComment  orElseP  parseEmpty
    parseDecl = Just <$> (
        (,) <*> parseIdent
            <*  manyP (charP ' ')
            <*  charP '='
            <*  manyP (charP ' ')
            <*> many1P (anyCharButP '\n')
            <*  charP '\n')
    parseComment =
        Nothing <$ charP '#'
                <* many1P (anyCharButP '\n')
                <* charP '\n'
    parseEmpty = Nothing <$ charP '\n'
Transforming that to a generic description is quite straightforward. We use primitive again to wrap letterOrDigitP:
descrINI :: Descr f => f INIFile
descrINI = many1 parseSection
  where
    parseSection =
        (,) <*  char '['
            <*> parseIdent
            <*  char ']'
            <*  newline
            <*> (catMaybes <$> many parseLine)
    parseIdent = many1 (primitive "alphanum" letterOrDigitP)
    parseLine = parseDecl  orElse  parseComment  orElse  parseEmpty
    parseDecl = Just <$> (
        (,) <*> parseIdent
            <*  many (char ' ')
            <*  char '='
            <*  many (char ' ')
            <*> many1 (primitive "non-newline" (anyCharButP '\n'))
	    <*  newline)
    parseComment =
        Nothing <$ char '#'
                <* many1 (primitive "non-newline" (anyCharButP '\n'))
		<* newline
    parseEmpty = Nothing <$ newline
This yields this not very helpful grammar (abbreviated here):
*Main> putStr $ ppGrammar2 "ini" descrINI
ini = '[', alphanum,  alphanum , ']', newline,  alphanum,  alphanum ,  ' ' 
But with a few uses of nonTerminal, we get something really nice:
descrINI :: Descr f => f INIFile
descrINI = many1 parseSection
  where
    parseSection = nonTerminal "section" $
        (,) <$  char '['
            <*> parseIdent
            <*  char ']'
            <*  newline
            <*> (catMaybes <$> many parseLine)
    parseIdent = nonTerminal "identifier" $
        many1 (primitive "alphanum" letterOrDigitP)
    parseLine = nonTerminal "line" $
        parseDecl  orElse  parseComment  orElse  parseEmpty
    parseDecl = nonTerminal "declaration" $ Just <$> (
        (,) <$> parseIdent
            <*  spaces
            <*  char '='
            <*  spaces
            <*> remainder)
    parseComment = nonTerminal "comment" $
        Nothing <$ char '#' <* remainder
    remainder = nonTerminal "line-remainder" $
        many1 (primitive "non-newline" (anyCharButP '\n')) <* newline
    parseEmpty = Nothing <$ newline
    spaces = nonTerminal "spaces" $ many (char ' ')
*Main> putStr $ ppGrammar "ini" descrINI
identifier = alphanum,  alphanum ;
spaces =  ' ' ;
line-remainder = non-newline,  non-newline , newline;
declaration = identifier, spaces, '=', spaces, line-remainder;
comment = '#', line-remainder;
line = declaration   comment   newline;
section = '[', identifier, ']', newline,  line ;
ini = section,  section ;

Recursion (variant 1) What if we want to write a parser/grammar-generator that is able to generate the following grammar, which describes terms that are additions and multiplications of natural numbers:
const = digit,  digit ;
spaces =  ' '   newline ;
atom = const   '(', spaces, expr, spaces, ')', spaces;
mult = atom,  spaces, '*', spaces, atom , spaces;
plus = mult,  spaces, '+', spaces, mult , spaces;
expr = plus;
The production of expr is recursive (via plus, mult, atom). We have seen above that simply defining a Grammar a recursively does not go well. One solution is to add a new combinator for explicit recursion, which replaces nonTerminal in the method:
class Applicative f => Descr f where
     
    recNonTerminal :: String -> (f a -> f a) -> f a
instance Descr Parser where
     
    recNonTerminal _ p = let r = p r in r
instance Descr Grammar where
     
    recNonTerminal = recNonTerminalG
recNonTerminalG :: String -> (Grammar a -> Grammar a) -> Grammar a
recNonTerminalG name f =
    let G (prods, rhs) = f (G ([], NonTerminal name))
    in G (prods ++ [(name, rhs)], NonTerminal name)
nonTerminal :: Descr f => String -> f a -> f a
nonTerminal name p = recNonTerminal name (const p)
runGrammer :: String -> Grammar a -> BNF
runGrammer main (G (prods, NonTerminal nt))   main == nt = prods
runGrammer main (G (prods, rhs)) = prods ++ [(main, rhs)]
The change in runGrammer avoids adding a pointless expr = expr production to the output. This lets us define a parser/grammar-generator for the arithmetic expressions given above:
data Expr = Plus Expr Expr   Mult Expr Expr   Const Integer
    deriving Show
mkPlus :: Expr -> [Expr] -> Expr
mkPlus = foldl Plus
mkMult :: Expr -> [Expr] -> Expr
mkMult = foldl Mult
parseExpr :: Descr f => f Expr
parseExpr = recNonTerminal "expr" $ \ exp ->
    ePlus exp
ePlus :: Descr f => f Expr -> f Expr
ePlus exp = nonTerminal "plus" $
    mkPlus <$> eMult exp
           <*> many (spaces *> char '+' *> spaces *> eMult exp)
           <*  spaces
eMult :: Descr f => f Expr -> f Expr
eMult exp = nonTerminal "mult" $
    mkPlus <$> eAtom exp
           <*> many (spaces *> char '*' *> spaces *> eAtom exp)
           <*  spaces
eAtom :: Descr f => f Expr -> f Expr
eAtom exp = nonTerminal "atom" $
    aConst  orElse  eParens exp
aConst :: Descr f => f Expr
aConst = nonTerminal "const" $ Const . read <$> many1 digit
eParens :: Descr f => f a -> f a
eParens inner =
    id <$  char '('
       <*  spaces
       <*> inner
       <*  spaces
       <*  char ')'
       <*  spaces
And indeed, this works:
*Main> putStr $ ppGrammar "expr" parseExpr
const = digit,  digit ;
spaces =  ' '   newline ;
atom = const   '(', spaces, expr, spaces, ')', spaces;
mult = atom,  spaces, '*', spaces, atom , spaces;
plus = mult,  spaces, '+', spaces, mult , spaces;
expr = plus;

Recursion (variant 2) Interestingly, there is another solution to this problem, which avoids introducing recNonTerminal and explicitly passing around the recursive call (i.e. the exp in the example). To implement that we have to adjust our Grammar type as follows:
newtype Grammar a = G ([String] -> (BNF, RHS))
The idea is that the list of strings is those non-terminals that we are currently defining. So in nonTerminal, we check if the non-terminal to be introduced is currently in the process of being defined, and then simply ignore the body. This way, the recursion is stopped automatically:
nonTerminalG :: String -> (Grammar a) -> Grammar a
nonTerminalG name (G g) = G $ \seen ->
    if name  elem  seen
    then ([], NonTerminal name)
    else let (prods, rhs) = g (name : seen)
         in (prods ++ [(name, rhs)], NonTerminal name)
After adjusting the other primitives of Grammar (including the Functor and Applicative instances, wich now again have nonTerminal) to type-check again, we observe that this parser/grammar generator for expressions, with genuine recursion, works now:
parseExp :: Descr f => f Expr
parseExp = nonTerminal "expr" $
    ePlus
ePlus :: Descr f => f Expr
ePlus = nonTerminal "plus" $
    mkPlus <$> eMult
           <*> many (spaces *> char '+' *> spaces *> eMult)
           <*  spaces
eMult :: Descr f => f Expr
eMult = nonTerminal "mult" $
    mkPlus <$> eAtom
           <*> many (spaces *> char '*' *> spaces *> eAtom)
           <*  spaces
eAtom :: Descr f => f Expr
eAtom = nonTerminal "atom" $
    aConst  orElse  eParens parseExp
Note that the recursion is only going to work if there is at least one call to nonTerminal somewhere around the recursive calls. We still cannot implement many as naively as above.

Homework If you want to play more with this: The homework is to define a parser/grammar-generator for EBNF itself, as specified in this variant:
identifier = letter,  letter   digit   '-' ;
spaces =  ' '   newline ;
quoted-char = non-quote-or-backslash   '\\', '\\'   '\\', '\'';
terminal = '\'',  quoted-char , '\'', spaces;
non-terminal = identifier, spaces;
option = '[', spaces, rhs, spaces, ']', spaces;
repetition = ' ', spaces, rhs, spaces, ' ', spaces;
group = '(', spaces, rhs, spaces, ')', spaces;
atom = terminal   non-terminal   option   repetition   group;
sequence = atom,  spaces, ',', spaces, atom , spaces;
choice = sequence,  spaces, ' ', spaces, sequence , spaces;
rhs = choice;
production = identifier, spaces, '=', spaces, rhs, ';', spaces;
bnf = production,  production ;
This grammar is set up so that the precedence of , and is correctly implemented: a , b c will parse as (a, b) c. In this syntax for BNF, terminal characters are quoted, i.e. inside ' ', a ' is replaced by \' and a \ is replaced by \\ this is done by the function quote in ppRHS. If you do this, you should able to round-trip with the pretty-printer, i.e. parse back what it wrote:
*Main> let bnf1 = runGrammer "expr" parseExpr
*Main> let bnf2 = runGrammer "expr" parseBNF
*Main> let f = Data.Maybe.fromJust . parse parseBNF. ppBNF
*Main> f bnf1 == bnf1
True
*Main> f bnf2 == bnf2
True
The last line is quite meta: We are using parseBNF as a parser on the pretty-printed grammar produced from interpreting parseBNF as a grammar.

Conclusion We have again seen an example of the excellent support for abstraction in Haskell: Being able to define so very different things such as a parser and a grammar description with the same code is great. Type classes helped us here. Note that it was crucial that our combined parser/grammars are only able to use the methods of Applicative, and not Monad. Applicative is less powerful, so by giving less power to the user of our Descr interface, the other side, i.e. the implementation, can be more powerful. The reason why Applicative is ok, but Monad is not, is that in Applicative, the results do not affect the shape of the computation, whereas in Monad, the whole point of the bind operator (>>=) is that the result of the computation is used to decide the next computation. And while this is perfectly fine for a parser, it just makes no sense for a grammar generator, where there simply are no values around! We have also seen that a phantom type, namely the parameter of Grammar, can be useful, as it lets the type system make sure we do not write nonsense. For example, the type of orElseG ensures that both grammars that are combined here indeed describe something of the same type.

  1. It seems to be the week of applicative-appraising blog posts: Brent has posted a nice piece about enumerations using Applicative yesterday.
  2. I like how in this alignment of <*> and <* the > point out where the arguments are that are being passed to the function on the left.

06 October 2016

Reproducible builds folks: Reproducible Builds: week 75 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday September 25 and Saturday October 1 2016: Statistics For the first time, we reached 91% reproducible packages in our testing framework on testing/amd64 using a determistic build path. (This is what we recommend to make packages in Stretch reproducible.) For unstable/amd64, where we additionally test for reproducibility across different build paths we are at almost 76% again. IRC meetings We have a poll to set a time for a new regular IRC meeting. If you would like to attend, please input your available times and we will try to accommodate for you. There was a trial IRC meeting on Friday, 2016-09-31 1800 UTC. Unfortunately, we did not activate meetbot. Despite this participants consider the meeting a success as several topics where discussed (eg changes to IRC notifications of tests.r-b.o) and the meeting stayed within one our length. Upcoming events Reproduce and Verify Filesystems - Vincent Batts, Red Hat - Berlin (Germany), 5th October, 14:30 - 15:20 @ LinuxCon + ContainerCon Europe 2016. From Reproducible Debian builds to Reproducible OpenWrt, LEDE & coreboot - Holger "h01ger" Levsen and Alexander "lynxis" Couzens - Berlin (Germany), 13th October, 11:00 - 11:25 @ OpenWrt Summit 2016. Introduction to Reproducible Builds - Vagrant Cascadian will be presenting at the SeaGL.org Conference In Seattle (USA), November 11th-12th, 2016. Previous events GHC Determinism - Bartosz Nitka, Facebook - Nara (Japan), 24th September, ICPF 2016. Toolchain development and fixes Michael Meskes uploaded bsdmainutils/9.0.11 to unstable with a fix for #830259 based on Reiner Herrmann's patch. This fixed locale_dependent_symbol_order_by_lorder issue in the affected packages (freebsd-libs, mmh). devscripts/2.16.8 was uploaded to unstable. It includes a debrepro script by Antonio Terceiro which is similar in purpose to reprotest but more lightweight; specific to Debian packages and without support for virtual servers or configurable variations. Packages reviewed and fixed, and bugs filed The following updated packages have become reproducible in our testing framework after being fixed: The following updated packages appear to be reproducible now for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.) Some uploads have addressed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Reviews of unreproducible packages 77 package reviews have been added, 178 have been updated and 80 have been removed in this week, adding to our knowledge about identified issues. 6 issue types have been updated: Weekly QA work As part of reproducibility testing, FTBFS bugs have been detected and reported by: diffoscope development A new version of diffoscope 61 was uploaded to unstable by Chris Lamb. It included contributions from: Post-release there were further contributions from: reprotest development A new version of reprotest 0.3.2 was uploaded to unstable by Ximin Luo. It included contributions from: Post-release there were further contributions from: tests.reproducible-builds.org Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

07 September 2016

Reproducible builds folks: Reproducible Builds: week 71 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday August 28 and Saturday September 3 2016: Media coverage Antonio Terceiro blogged about testing build reprodubility with debrepro . GSoC and Outreachy updates The next round is being planned now: see their page with a timeline and participating organizations listing. Maybe you want to participate this time? Then please reach out to us as soon as possible! Packages reviewed and fixed, and bugs filed The following packages have addressed reproducibility issues in other packages: The following updated packages have become reproducible in our current test setup after being fixed: The following updated packages appear to be reproducible now, for reasons we were not able to figure out yet. (Relevant changelogs did not mention reproducible builds.) The following 4 packages were not changed, but have become reproducible due to changes in their build-dependencies: Some uploads have addressed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Reviews of unreproducible packages 706 package reviews have been added, 22 have been updated and 16 have been removed in this week, adding to our knowledge about identified issues. 5 issue types have been added: 1 issue type has been updated: Weekly QA work FTBFS bugs have been reported by: diffoscope development diffoscope development on the next version (60) continued in git, taking in contributions from: strip-nondeterminism development Mattia Rizzolo uploaded strip-nondeterminism 0.023-2~bpo8+1 to jessie-backports. A new version of strip-nondeterminism 0.024-1 was uploaded to unstable by Chris Lamb. It included contributions from: Holger added jobs on jenkins.debian.net to run testsuites on every commit. There is one job for the master branch and one for the other branches. disorderfs development Holger added jobs on jenkins.debian.net to run testsuites on every commit. There is one job for the master branch and one for the other branches. tests.reproducible-builds.org Debian: We now vary the GECOS records of the two build users. Thanks to Paul Wise for providing the patch. Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

27 July 2016

Norbert Preining: TUG 2016 Day 2 Figures to Fonts

The second day of TUG 2016 was again full of interesting talks spanning from user experiences to highly technical details about astrological chart drawing, and graphical user interfaces to TikZ to the invited talk by Robert Bringhurst on the Palatino family of fonts. tug2016-bringhurst With all these interesting things there is only one thing to compain I cannot get out of the dark basement and enjoy the city After a evening full of sake and a good night s sleep we were ready to dive into the second day of TUG. Kaveh Bazargan A graphical user interface for TikZ The opening speaker of Day 2 was Kaveh. He first gave us a quick run-down on what he is doing for business and what challenges publishers are facing in these times. After that he introduced us to his new development of a command line graphical user interface for TikZ. I wrote command line on purpose, because the editing operations are short commands issued on a kind of command line, which will give an immediate graphical feedback. Basic of the technique is a simplified TikZ-like meta language that is not only easy to write, but also easy to parse. While the amount of supported commands and features of TikZ is still quite small, I think the basic idea is a good one, and there is a good potential in it. Matthew Skala Astrological charts with horoscop and starfont Next up was Matthew who introduced us to the involved task of typesetting astrological charts. He included comparisons with various commercial and open source solutions, where Matthew of course, but me too, felt that his charts came of quite well! As an extra bonus we got some charts of famous singers, as well as the TUG 2016 horoscope. David Tulett Development of an e-textbook using LaTeX and PStricks David reported on his project to develop an e-textbook on decision modeling (lots of math!) using LaTeX and PStricks. His e-book is of course a PDF. There were a lot of very welcoming feedback free (CC-BY-NC-ND) textbooks for sciences are rare and we need more of them. Christian Gagn An Emacs-based writing workflow inspired by TeX and WEB, targeting the Web Christian s talk turned around editing and publishing using org-mode of Emacs and the various levels of macros one can use in this setup. He finished with a largely incomprehensible vision of a future equational logic based notation mode. I have used equational logic in my day-in-day-out job, and I am not completely convinced that this is a good approach for typesetting and publishing but who knows, I am looking forward to a more logic-based approach! Barbara Beeton, Frank Mittelbach In memoriam: Sebastian Rahtz (1955-2016) Frank recalled Sebastian s many contribution to a huge variety of fields, and recalled our much missed colleague with many photos and anecdotes. Jim Hefferon A LaTeX reference manual Jim reported about the current state of a LaTeX reference manual, which tries to provide a documentation orthogonally to the many introduction and user guides available, by providing a straight down-to-earth reference manual with all the technical bells and whistles necessary. As I had to write myself a reference manual for a computer language, it was very interested to see how they dealt with many of the same problems I am facing. Arthur Reutenauer, Mojca Miklavec Hyphenation past and future: hyph-utf8 and patgen Arthur reports about the current statue of the hyphenation pattern project, and in particular the license and usage hell they recently came into with large cooperations simply grabbing the patterns without proper attribution. In a second part he gave a rough sketch of his shot at a reimplementation of patgen. Unfortunately he wrote in rather unreadable hand-writing on a flip-chart, which made only the first line audience to actually see what he was writing. Federico Garcia-De Castro TeXcel? As an artist organizing large festivals Federico has to fight with financial planning and reports. He seemed not content with the abilities of the usual suspects, so he developed a way to do Excel like book-keeping in TeX. Nice idea, I hope I can use that system for the next conference I have to organize! Jennifer Claudio A brief reflection on TeX and end-user needs Last speaker in the morning session was Jennifer who gave us a new and end-user s view onto the TeX environment, and the respective needs. These kind of talks are a very much welcomed contrast to technical talks and hopefully all of us developers take home some of her suggestions. Sungmin Kim, Jaeyoung Choi, Geunho Jeong MFCONFIG: Metafont plug-in module for the Freetype rasterizer Jaeyoung reported about an impressive project to make Metafont fonts available to fontconfig and thus windowing systems. He also explained their development of a new font format Stemfont, which is a Metafont-like system that can work also for CJK fonts, and which they envisage to be built into all kind of mobile devices. Michael Sharpe New font offerings Cochineal, Nimbus15 and LibertinusT1Math Michael reports about his last font projects. The first two being extensions of the half-made half-butchered rereleased URW fonts, as well as his first (?) math font project. I talked to him over lunch one day, and asked him how many man-days he need for these fonts, and his answer was speaking a lot: For the really messed up new URW fonts, like Cochineal, he guessed about 5 man-months of work, while other fonts only needed a few days. I think we all can be deeply thankful to all the work he is investing into all these font projects. Robert Bringhurst The evolution of the Palatino tribe The second invited talk was Robert Bringhurst, famous for his wide contributions to typpography, book culture in general, as well as poetry. He gave a quick historic overview on the development of the Palatino tribe of fonts, with lots of beautiful photos. I was really looking forward to Robert s talk, and my expectations were extremely high. And unfortunately I must say I was quite disappointed. Maybe it is his style of presentation, but the feeling he transfered to me (the audience?) was that he was going through a necessary medical check, not much enjoying the presentation. Also, the content itself was not really full of his own ideas or thoughts, but a rather superficial listing of historical facts. Of course, a person like Robert Bringhurst is so full of anecdotes and background knowledge still was a great pleasure to listen and lots of things to learn, I only hoped for a bit more enthusiasm. TUG Annual General Meeting The afternoon session finished with the TUG Annual General Meeting, reports will be sent out soon to all TUG members. Herbert Schulz Optional workshop: TeXShop tips & tricks After the AGM, Herbert from MacTeX and TeXShop gave an on-the-spot workshop on TeXShop. Since I am not a Mac user, I skipped on that.
Another late afternoon program consisted of an excursion to Eliot s bookshop, where many of us stacked up on great books. This time again I skipped and took a nap. In the evening we had a rather interesting informal dinner in the food court of some building, where only two shops were open and all of us lined up in front of the Japanese Curry shop, and then gulped down from plastic boxes. Hmm, not my style I have to say, not even for informal dinner. But at least I could meet up with a colleague from Debian and get some gpg key signing done. And of course, talking to all kind of people around. The last step for me was in the pub opposite the hotel, with beer and whiskey/scotch selected by specialists in the field.

26 July 2016

Norbert Preining: TUG 2016 Day 1 Routers and Reading

The first day of the real conference started with an excellent overview of what one can do with TeX, spanning from traditional scientific journal styles to generating router configuration for cruising ships.
tug2016-color All this was crowned with an invited talk my Kevin Larson from Microsoft s typography department on how to support reading comprehension. Pavneet Aurora Opening: Passport to the TeX canvas Pavneet, our never-sleeping host and master of organization, opened the conference with a very philosophical introduction, touching upon a wide range of topics ranging from Microsoft, Twitter to the beauty of books, pages, and type. I think at some point he even mentioned TeX, but I can t remember for sure. His words put up a very nice and all-inclusive stage, a community that is open to all kind of influences with any disregard or prejudice. Let us hope that is reflects reality. Thanks Pavneet. Geoffrey Poore Advances in PythonTeX Our first regular talk was by Geoffrey reporting on recent advances in PythonTeX, a package that allows including python code in your TeX document. Starting with an introduction to PythonTeX, Geoggrey reports about an improved verbatim environment, fvextra, which patches fancyvrb, and improved interaction between tikz and PythonTeX. As I am a heavy user of listings for my teaching on algebraic specification languages, I will surely take a look at this package and see how it compares to listings. Stefan Kottwitz TeX in industry I: Programming Cisco network switches using TeX Next was Stefan from Lufthansa Industry Solutions, who reported first about his working environment, Cruise Ships with a very demanding IT infrastructure he has to design and implement. Then he introduced us to his way of generating IP configurations for all the devices using TeX. The reason he chose this method is that it allows him to generate at the same time proper documentation. It was surprising for me to hear that by using TeX he could far more efficiently and quicker produce well designed and easily accessible documentation, which helped both the company as well as made the clients happy! Stefan Kottwitz TeX in industry II: Designing converged network solutions After a coffee break, Stefan continued his exploration into industrial usage of TeX, this time about using tikz to generate graphics representing the network topology on the ships. Boris Veytsman Making ACM LaTeX styles Next up was Boris which brought us back to traditional realms of TeX when he guided us into the abyss of ACM LaTeX styles he tried to maintain for some time, until he plunged into a complete rewrite of the styles. Frank Mittelbach Alice goes floating global optimized pagination including picture placements The last talk before lunch (probably a strategic placement, otherwise Frank would continue for hours and hours) was Frank on global optimization of page breaks. Frank showed us what can and can not be done with current LaTeX, and how to play around with global optimization of pagination, using Alice in Wonderland as running example. We can only hope that his package is soon available in an easily consumable version to play around. Thai lunch Pavneet has organized three different lunch-styles for the three days of the conference, today s was Thai with spring rools, fried noodles, one kind of very interesting orange noodles, and chicken something. Michael Doob baseball rules summary After lunch Michael gave us an accessible explanation of the most arcane rules a game can have the baseball rules by using pseudo code. I think the total number of loc needed to explain the overall rules would fill more pages than the New York phonebook, so I am deeply impressed by all those who can understand these rules. Some of us even wandered off in the late afternoon to see a match with life explanations of Michael. Amartyo Banerjee, S.K. Venkatesan A Telegram bot for printing LaTeX files Next up was Amartyo who showed a Telegram (as in messenger application) bot running on a Raspberry Pi, that receives (La)TeX files and sends back compiled PDF files. While it is not ready for consumption (If you sneeze the bot will crash!), it looks like a promising application. Furthermore, it is nice to see how open APIs (like Telegram) can spur development of useful tools, while closed APIs (including threatening users, like WhatApp) hinders it. Norbert Preining Security improvements in the TeX Live Manager and installer Next up was my own talk about beefing up the security of TeX Live by providing integrity and authenticity checks via GnuPG, a feature that has been introduced with the recent release of TeX Live 2016. The following discussion gave me several good idea on how to further improve security and usability. Arthur Reutenauer -The TeX Live M sub-project (and open discussion) Arthur presented the TeX Live M (where the M stands supposedly for Mojca, who couldn t attend unfortunately) project: Their aim is to provide a curated and quality verified sub-part of TeX Live that is sufficiently complete for many applications, and easier for distributors and packagers. We had a lively discussion after Arthur s short presentation, mostly about why TeX Live does not have a on-the-fly installation like MikTeX. I insisted that this is already possible, using the tex-on-the-fly package which uses the mktextex infrastructure, but also caution against using it by default due to delays induced by repeatably reading the TeX Live database. I think this is a worth-while project for someone interested in learning the internals of TeX Live, but I am not sure whether I want to invest time into this feature. Another discussion point was about a testing infrastructure, which I am currently working on. This is in fact high on my list, to have some automatic minimal functionality testing a LaTeX package should at least load! Kevin Larson Reading between the lines: Improving comprehension for students Having a guest from Microsoft is rather rare in our quite Unix-centered environment, so big thanks to Pavneet again for setting up this contact, and big thanks to Kevin for coming. Kevin gave us a profound introduction to reading disabilities and how to improve reading comprehension. Starting with an excursion into what makes a font readable and how Microsoft develops optimally readable fonts, he than turned to reading disabilities like dyslexia, and how markup of text can increase students comprehension rate. He also toppled my long-term believe that dyslexia is connected to the similar shape of letters which are somehow visually malprocessed this was the scientific status from the 1920ies till the 70ies, but since then all researchers have abandoned this interpretation and dyslexia is now linked to problems linking shape to phonems. Kevin did an excellent job with a slightly difficult audience some people picking about grammer differences between British and US English and permanently derailing the discussion, and even more the high percentage of typographically somehow specially tasted participants. After the talk I had a lengthy discussion with Kevin about if/how this research can be carried over to non-Roman writing systems, in particular Kanji/Hanzi based writing systems, where dyslexia shows itself probably in different context. Kevin also mentioned that they want to add interword space to Chinese to help learners of Chinese (children, foreigners) to better parse, and studies showed that this helps a lot in comprehension. On a meta-level, this talk bracketed with the morning introduction by Pavneet, describing an open environment with stimulus back and forth in all directions. I am very happy that Kevin took the pain to come in his tight schedule, and I hope that the future will bring better cooperation at the end we are all working somehow on the same front only the the tools differ.
izakaya-sake-partyAfter the closing of the session, one part of our group went off to the baseball match, while another group dived into a Japanese-style Izakaya where we managed to kill huge amounts of sake and quite an amount of food. The photo shows me after the first bottle of sake, while just seeping on an intermediate small amount of genshu (kind of strong undiluted sake) before continuing to the next bottle. An interesting and stimulating first day of TUG, and I am sure that everyone was looking forward to day 2.

10 July 2016

Ritesh Raj Sarraf: Leprosy in India

During my recent travel, I had quite a long layover at the Doha International Airport in Qatar. While killing time, I watched an interesting programme on the Al Jazeera network. The program aired on Al Jazeera is Lifelines. This special episode I watched, covered about "Leprosy in India". After having watched the entire programme, I felt the urge to blog about it. First of all, it was quite depressing to me, to know through the programme, that the Govt. of India had officially marked "Leprosy" eradicated from India in 2005. As per Wikipedia, "Globally in 2012, the number of chronic cases of leprosy was 189,000, down from some 5.2 million in the 1980s. The number of new cases was 230,000. Most new cases occur in 16 countries, with India accounting for more than half." Of the data presented on Lifelines, they just covered a couple of districts from 2 States (of the 28+ States) of India. So, with many states remaining, and unserveyed, and uncounted, we are far away from making such statements. Given that the Govt., on paper, has marked Leprosy eradicated; so does WHO. Which means that there is no more funding to help people suffering from the disease. And with no Govt. (or International) funding, it is a rather disappointing situation. I come from a small town on the Indo-Nepal intl. border, named Raxaul. And here, I grew up seeing many people who suffered from Leprosy. As a child, I never much understood the disease, because as is mentioned in the programme, I was told it was a communicable disease. Those suffering, were (and are) tagged as Untouchables (Hindi: ). Of my small town, there was and still is a sub-town, named Sunderpur. This town houses patients suffering from Leprosy, from around multiple districts closeby. I've never been there, but have been told that it is run by an NGO, and helps patients fight the disease. Lifelines also reported that fresh surveys done by the Lepra society, just a 3 day campaign, resulted in 3808 new cases of people suffering from Leprosy. That is a big number, given accessibility to small towns only happens once a week on market day. And in 3 days the team hardly covered a couple of district towns. My plea to the Media Houses of India. Please spend some time to look beyond the phony stuff that you mostly present. There are real issues that could be brought to a wider audience. As for the government, it is just depressing to know how rogue some/most of your statements are.

Categories:

Keywords:

Like:

Ritesh Raj Sarraf: Leprosy in India

During my recent travel, I had quite a long layover at the Doha International Airport in Qatar. While killing time, I watched an interesting programme on the Al Jazeera network. The program aired on Al Jazeera is Lifelines. This special episode I watched, covered about "Leprosy in India". After having watched the entire programme, I felt the urge to blog about it. First of all, it was quite depressing to me, to know through the programme, that the Govt. of India had officially marked "Leprosy" eradicated from India in 2005. As per Wikipedia, "Globally in 2012, the number of chronic cases of leprosy was 189,000, down from some 5.2 million in the 1980s. The number of new cases was 230,000. Most new cases occur in 16 countries, with India accounting for more than half." Of the data presented on Lifelines, they just covered a couple of districts from 2 States (of the 28+ States) of India. So, with many states remaining, and unserveyed, and uncounted, we are far away from making such statements. Given that the Govt., on paper, has marked Leprosy eradicated; so does WHO. Which means that there is no more funding to help people suffering from the disease. And with no Govt. (or International) funding, it is a rather disappointing situation. I come from a small town on the Indo-Nepal intl. border, named Raxaul. And here, I grew up seeing many people who suffered from Leprosy. As a child, I never much understood the disease, because as is mentioned in the programme, I was told it was a communicable disease. Those suffering, were (and are) tagged as Untouchables (Hindi: ). Of my small town, there was and still is a sub-town, named Sunderpur. This town houses patients suffering from Leprosy, from around multiple districts closeby. I've never been there, but have been told that it is run by an NGO, and helps patients fight the disease. Lifelines also reported that fresh surveys done by the Lepra society, just a 3 day campaign, resulted in 3808 new cases of people suffering from Leprosy. That is a big number, given accessibility to small towns only happens once a week on market day. And in 3 days the team hardly covered a couple of district towns. My plea to the Media Houses of India. Please spend some time to look beyond the phony stuff that you mostly present. There are real issues that could be brought to a wider audience. As for the government, it is just depressing to know how rogue some/most of your statements are.

Categories:

Keywords:

Like:

26 April 2016

Reproducible builds folks: Reproducible builds: week 52 in Stretch cycle

What happened in the Reproducible Builds effort between April 17th and April 23rd 2016: Toolchain fixes Thomas Weber uploaded lcms2/2.7-1 which will not write uninitialized memory when writing color names. Original patch by Lunar. The GCC 7 development phase has just begun, so Dhole reworked his patch to make gcc use SOURCE_DATE_EPOCH if set which prompted interesting feedback, but it has not been merged yet. Alexis Bienven e submitted a patch for sphinx to strip Python object memory addresses from the generated documentation. Packages fixed The following packages have become reproducible due to changes in their build dependencies: cobertura, commons-pool, easymock, eclipselink, excalibur-logkit, gap-radiroot, gluegen2, jabref, java3d, jcifs, jline, jmock2, josql, jtharness, libfann, libgroboutils-java, libjemmy2-java, libjgoodies-binding-java, libjgrapht0.8-java, libjtds-java, liboptions-java, libpal-java, libzeus-jscl-java, node-transformers, octave-msh, octave-secs2d, openmama, rkward. The following packages have become reproducible after being fixed: Patches submitted that have not made their way to the archive yet: tests.reproducible-builds.org diffoscope development diffoscope 52 was released with changes from Mattia Rizzolo, h01ger, Satyam Zode and Reiner Herrmann, who also did the release. Notable changes included: As usual, diffoscope 52 is available on Debian, Archlinux and PyPI, other distributions will hopefully soon update. Package reviews 28 reviews have been added, 11 have been updated and 94 have been removed in this week. 14 FTBFS bugs were reported by Chris Lamb (one being was a duplicate of a bug filed by Sebastian Ramacher an hour earlier). Misc. This week's edition was written by Lunar, Holger 'h01ger' Levsen and Chris Lamb and reviewed by a bunch of Reproducible builds folks on IRC.

18 April 2016

Reproducible builds folks: Reproducible builds: week 50 in Stretch cycle

What happened in the reproducible builds effort between April 3rd and April 9th 2016: Media coverage Emily Ratliff wrote an article for SecurityWeek called Establishing Correspondence Between an Application and its Source Code - How Combining Two Completely Separate Open Source Projects Can Make Us All More Secure. Tails have started work on a design for freezable APT repositories to make it easier and practical to perform reproductions of an entire distribution at a given point in time, which will be needed to create reproducible installation- or live-media. Toolchain fixes Alexis Bienven e submitted patches adding support for SOURCE_DATE_EPOCH in several tools: transfig, imagemagick, rdtool, and asciidoctor. boyska submitted one for python-reportlab. Packages fixed The following packages have become reproducible due to changes in their build dependencies: atinject-jsr330 brailleutils cglib3 gnugo libcobra-java libgnumail-java libjchart2d-java libjcommon-java libjfreechart-java libjide-oss-java liblaf-widget-java liblastfm-java liboptions-java octave-control octave-mpi octave-nan octave-parallel octave-stk octave-struct octave-tsa oar The following packages became reproducible after getting fixed: Several uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet: Other upstream fixes Alexander Batischev made a commit to make newsbeuter reproducible. tests.reproducible-builds.org Package reviews 93 reviews have been removed, 66 added and 21 updated in the previous week. 12 new FTBFS bugs have been reported by Chris Lamb and Niko Tyni. Misc. This week's edition was written by Lunar, Holger Levsen, Reiner Herrmann, Mattia Rizzolo and Ximin Luo. With the departure of Lunar as a full-time contributor, Reproducible Builds Weekly News (this thing you're reading) has moved from his personal Debian blog on Debian People to the Reproducible Builds team web site on Debian Alioth. You may want to update your RSS or Atom feeds. Very many thanks to Lunar for writing and publishing this weekly news for so long, well & continously!

Next.