Joey Hess: Template Haskell on impossible architectures
Imagine you had an excellent successful Kickstarter campaign, and during
it a lot of people asked for an Android port to be made of the software.
Which is written in Haskell. No problem, you'd think -- the user interface
can be written as a local webapp, which will be nicely platform agnostic
and so make it easy to port. Also, it's easy to promise a lot of stuff
during a Kickstarter campaign. Keeps the graph going up. What could go
wrong?
So, rather later you realize there is no Haskell compiler for Android. At
all. But surely there will be eventually. And so you go off and build the
webapp. Since Yesod seems to be the pinnacle of type-safe Haskell web
frameworks, you use it. Hmm, there's this Template Haskell stuff that it
uses a lot, but it only makes compiles a little slow, and the result is
cool, so why not.
Then, about half-way through the project, it seems time to get around
to this Android port. And, amazingly, a
Haskell compiler for Android
has appeared in the meantime. Like the Haskell community has your back.
(Which they generally seem to.) It's early days and rough, lots of libraries
need to be hacked to work, but it only takes around 2 weeks to get a
port of your program that basically works.
But, no webapp. Cause nobody seems to know how to make a cross-compiling
Haskell compiler do Template Haskell. (Even building a fully native
compiler on Android doesn't do the trick. Perhaps you missed something
though.)
At this point you can give up and write a separate Android UI (perhaps
using these new Android JNI bindings for Haskell that have also
appeared in the meantime). Or you can procrastinate for a while, and mull
it over; consider rewriting the webapp to not use Yesod but some other
framework that doesn't need Template Haskell.
Eventually you might think this: If I run ghc -ddump-splices when I'm
building my Yesod code, I can see all the thousands of lines of delicious
machine generated Haskell code. I just have to paste that in, in
place of the Template Haskell that generated it, and I'll get a program I
can build on Android! What could go wrong?
And you even try it, and yeah, it seems to work. For small amounts of code
that you paste in and carefully modify and get working. Not a whole
big, constantly improving webapp where every single line of html gets
converted to types and lambdas that are somehow screamingly fast.
So then, let's automate this pasting.
And so the EvilSplicer is born!
That's a fairly general-purpose Template Haskell splicer. First do a native
build with -ddump-splices output redirected to a log file. Run the
EvilSplicer to fix up the code. Then run an Android cross-compile.
But oh, the caveats. There are so many ways this can go wrong..
- The first and most annoying problem you'll encounter is that often Template Haskell splices refer to hidden symbols that are not exported from the modules that define the splices. This lets the splices use those symbols, but prevents them being used in your code. This does not seem like a good part of the Template Haskell design, to be honest. It would be better if it required all symbols used in splices to be exported. But it can be worked around. Just use trial and error to find every Haskell library that does this, and then modify them to export all the symbols they use. And after each one, rebuild all libraries that depend on it. You're very unlikely to end up with more than 9 thousand lines of patches. Because that's all it took me..
- The next problem (and the next one, and the next ...) is that
while GHC's code output by
-dump-splices(and indeed, by GHC error messages, etc) looks like valid Haskell code to the casual viewer, it's often not. To start with, it often has symbols qualified with the package and module name.ghc-prim:GHC.Types.:does not work well where code originally contained:. And then there's fun with multi-line strings, which sometimes cannot be parsed back in by GHC in the form it outputs them. And then there's the strange way GHC outputscaseexpressions, which is not valid Haskell at all. (It's missing some semicolons.) Oh, and there's the lambda expressions that GHC outputs with insufficient parentheses, leading to type errors at compile time. And so much more fun. Enough fun to give one the idea that this GHC output has never really been treated as code that could be run again. Because that would be a dumb thing to need to do. - Just to keep things interesting, the Haskell libraries used by your native GHC and your Android GHC need to be pretty much identical versions. Maybe a little wiggle room, but any version skew could cause unwanted results. Probably, most of the time, unwanted results in the form of a 3 screen long type error message. (My longest GHC error message seen on this odyessy was actually a full 500+ kilobytes in size. It included the complete text of Jquery and Bootstrap. At times like these you notice that GHC outputs its error messages o.n.e . c.h.a.r.a.c.t.e.r . a.t . a . t.i.m.e.)
PS, I hope nobody will ever find this blog post useful in their work. PPS, Also, if you let this put you off Haskell in any way .. well, don't. You just might want to wait a year or so before doing Haskell on Android.
na ir slavena ne tikai ar savu akmens m ri valsts zieme os, bat ar ar Di o nas Ugunsm ri apk rt visam s valsts Internetam, kas blo visu p c k rtas un iebremzina visu p r jo. Man pirm person g saskarsme ar o pakalpojumu notika jau anhajas lidost , kad tri vien izr d j s, ka valst blo ts ir ne tikai Facebook, bet ar Twitter, kas iev rojami apgr tin ja manas iesp jas tri un viegli apzi ot visus, ka es esmu v l joproj m esmu dz vs un vesels. P c p ris eksperimentiem izr d j s, ka, lai ar no telefona nav pieejama Google+ m jas lapa un nav lejupl d jama Google+ (un WhatsApp) programma uz Android, tom r, ja t s jau ir telefon , ie abi servisi turpina no telefona str d t. T p c es s ku rakst t ce ojuma piez mes Google+ un da as dienas p c ce ojuma s kuma man pat izdev s nokonfigur t
I've only helped
"politically" here and there with this over the years, and I'm
happy to see it live. Your thanks for this should go to the blog
editors---Francesca Ciceri and Ana Guerrero Lopez---and to DSA for
making it real. A proper delegation for the editors is pending and
I'm confident the next DPL will pick it up.
After nearly one year of development, the "Port bootstrap build-ordering tool"
now finally has a name: botch. It stands for Bootstrap/Build Order Tool CHain.
Now we dont have to call it "Port bootstrap build-ordering tool" anymore
(nobody did that anyways) and also dont have to talk about it by referring to
it as "the tools" in email, IRC or in my master thesis which is due in a few
weeks. With this, the issue also doesnt block the creation of a Debian package
anymore. Since only a handful of people have a clone of it anyway, I also
renamed the gitorious git repository url (and updated all links in this blog
and left a text file informing about the name change in the old location). The
new url is:
I am led to believe that splunkd (some agent for feeding log entries into
the Grand Log Analysis Tool Of Our Age ) has no capability for running
itself in the foreground. This is stupid. Do not make these sorts of
assumptions about how the user will want to run your software. Some people
use sane service-management systems that are capable of handling the
daemonisation for you and automatically restart the managed process on
crash. These systems are typically much easier to configure and debug, and
they don t need bloody PID files and the arguments about where to put them
(tmpfs, inside or outside chroots oh my) and who should update them and
how to reliably detect that they re out of date when they crash without
causing race conditions and whether non-root-running processes should place
their PID files in the same place and how do you deal with the permissions
issues and bugger that for a game of skittles.
In short, if you provide a service daemon and do not provide some
well-documented means of saying don t background , I will hurt you. This
goes double if your shitware is not open source.
2 weeks passed, & I worked on exactly 2 RC bugs *cough*
Disclaimer
I'm no symbol expert and dealing with symbol files for first time from
the time I started packaging. What I did here is depending on some
suggestions I got in #debian-mentors.
If you think what I did was wrong please enlighten me :-).
Problem
Recently 2 of my library packages pugixml and ctpp2 got accepted
into Debian archive and when buildd tried to build them on remaining
architectures other than one for which I uploaded (amd64) builds
failed. This was expected as symbols file which I generated was for
amd64. As usual I got 2 serious bug reports

As previously mentioned I was looking to package pwsafe for Wheezy, as this is one of the few tools that I rely upon which isn't present.
There are now