Brief Glamor Hacks
Eric Anholt started writing Glamor a few years ago. The goal was to
provide credible 2D acceleration based solely on the OpenGL API, in
particular, to implement the X drawing primitives, both core and
Render extension, without any GPU-specific code. When he started,
the thinking was that fixed-function devices were still relevant, so
that original code didn t insist upon modern OpenGL features like
GLSL shaders. That made the code less efficient and hard to write.
Glamor used to be a side-project within the X world; seen as something
that really wasn t very useful; something that any credible 2D driver
would replace with custom highly-optimized GPU-specific code. Eric and
I both hoped that Glamor would turn into something credible and that
we d be able to eliminate all of the horror-show GPU-specific code in
every driver for drawing X text, rectangles and composited
images. That hadn t happened though, until now.
Fast forward to the last six months. Eric has spent a bunch of time
cleaning up Glamor internals, and in fact he s had it merged into the
core X server for version 1.16 which will be coming up this July.
Within the Glamor code base, he s been cleaning some internal
structures up and making life more tolerable for Glamor developers.
Using libepoxy
A big part of the cleanup was a transition all of the extension
function calls to use his other new project,
libepoxy, which provides a sane,
consistent and performant API to OpenGL extensions for Linux, Mac OS
and Windows. That library is awesome, and you should use it for
everything you do with OpenGL because not using it is like pounding
nails into your head. Or eating non-tasty pie.
Using VBOs in Glamor
One thing he recently cleaned up was how to deal with VBOs
during X operations. VBOs are absolutely essential to modern OpenGL
applications; they re really the only way to efficiently pass vertex
data from application to the GPU. And, every other mechanism is
deprecated by the ARB as not a part of the blessed core context .
Glamor provides a simple way of getting some VBO space, dumping data
into it, and then using it through two wrapping calls which you use
along with glVertexAttribPointer as follows:
pointer = glamor_get_vbo_space(screen, size, &offset);
glVertexAttribPointer(attribute_location, count, type,
GL_FALSE, stride, offset);
memcpy(pointer, data, size);
glamor_put_vbo_space(screen);
glamor_get_vbo_space allocates the specified amount of VBO space
and returns a pointer to that along with an offset , which is
suitable to pass to glVertexAttribPointer. You dump your data into the
returned pointer, call glamor_put_vbo_space and you re all done.
Actually Drawing Stuff
At the same time, Eric has been optimizing some of the existing
rendering code. But, all of it is still frankly terrible. Our dream of
credible 2D graphics through OpenGL just wasn t being realized at all.
On Monday, I decided that I should go play in Glamor for a few days,
both to hack up some simple rendering paths and to familiarize myself
with the insides of Glamor as I m getting asked to review piles of
patches for it, and not understanding a code base is a great way to
help introduce serious bugs during review.
I started with the core text operations. Not because they re
particularly relevant these days as most applications draw text with
the Render extension to provide better looking results, but instead
because they re often one of the hardest things to do efficiently with
a heavy weight GPU interface, and OpenGL can be amazingly heavy weight
if you let it.
Eric spent a bunch of time optimizing the existing text code to try
and make it faster, but at the bottom, it actually draws each lit
pixel as a tiny GL_POINT object by sending a separate x/y vertex value
to the GPU (using the above VBO interface). This code walks the array
of bits in the font and checking each one to see if it is lit, then
checking if the lit pixel is within the clip region and only then
adding the coordinates of the lit pixel to the VBO. The amazing thing
is that even with all of this CPU and GPU work, the venerable 6x13
font is drawn at an astonishing 3.2 million glyphs per second. Of
course, pure software draws text at 9.3 million glyphs per second.
I suspected that a more efficient implementation might be able to draw
text a bit faster, so I decided to just start from scratch with a new
GL-based core X text drawing function. The plan was pretty simple:
- Dump all glyphs in the font into a texture. Store them in 1bpp
format to minimize memory consumption.
- Place raw (integer) glyph coordinates into the VBO. Place
four coordinates for each and draw a GL_QUAD for each glyph.
- Transform the glyph coordinates into the usual GL range (-1..1)
in the vertex shader.
- Fetch a suitable byte from the glyph texture, extract a single bit
and then either draw a solid color or discard the fragment.
This makes the X server code surprisingly simple; it computes integer
coordinates for the glyph destination and glyph image source and
writes those to the VBO. When all of the glyphs are in the VBO, it
just calls glDrawArrays(GL_QUADS, 0, 4 * count). The results were
encouraging :
1: fb-text.perf
2: glamor-text.perf
3: keith-text.perf
1 2 3 Operation
------------ ------------------------- ------------------------- -------------------------
9300000.0 3160000.0 ( 0.340) 18000000.0 ( 1.935) Char in 80-char line (6x13)
8700000.0 2850000.0 ( 0.328) 16500000.0 ( 1.897) Char in 70-char line (8x13)
6560000.0 2380000.0 ( 0.363) 11900000.0 ( 1.814) Char in 60-char line (9x15)
2150000.0 700000.0 ( 0.326) 7710000.0 ( 3.586) Char16 in 40-char line (k14)
894000.0 283000.0 ( 0.317) 4500000.0 ( 5.034) Char16 in 23-char line (k24)
9170000.0 4400000.0 ( 0.480) 17300000.0 ( 1.887) Char in 80-char line (TR 10)
3080000.0 1090000.0 ( 0.354) 7810000.0 ( 2.536) Char in 30-char line (TR 24)
6690000.0 2640000.0 ( 0.395) 5180000.0 ( 0.774) Char in 20/40/20 line (6x13, TR 10)
1160000.0 351000.0 ( 0.303) 2080000.0 ( 1.793) Char16 in 7/14/7 line (k14, k24)
8310000.0 2880000.0 ( 0.347) 15600000.0 ( 1.877) Char in 80-char image line (6x13)
7510000.0 2550000.0 ( 0.340) 12900000.0 ( 1.718) Char in 70-char image line (8x13)
5650000.0 2090000.0 ( 0.370) 11400000.0 ( 2.018) Char in 60-char image line (9x15)
2000000.0 670000.0 ( 0.335) 7780000.0 ( 3.890) Char16 in 40-char image line (k14)
823000.0 270000.0 ( 0.328) 4470000.0 ( 5.431) Char16 in 23-char image line (k24)
8500000.0 3710000.0 ( 0.436) 8250000.0 ( 0.971) Char in 80-char image line (TR 10)
2620000.0 983000.0 ( 0.375) 3650000.0 ( 1.393) Char in 30-char image line (TR 24)
This is our old friend x11perfcomp, but slightly adjusted for a modern
reality where you really do end up drawing billions of objects (hence
the wider columns). This table lists the performance for drawing a
range of different fonts in both poly text and image text
variants. The first column is for Xephyr using software (fb)
rendering, the second is for the existing Glamor GL_POINT based code
and the third is the latest GL_QUAD based code.
As you can see, drawing points for every lit pixel in a glyph is
surprisingly fast, but only about 1/3 the speed of software for
essentially any size glyph. By minimizing the use of the CPU and
pushing piles of work into the GPU, we manage to increase the speed of
most of the operations, with larger glyphs improving significantly
more than smaller glyphs.
Now, you ask how much code this involved. And, I can truthfully say
that it was a very small amount to write:
Makefile.am 2
glamor.c 5
glamor_core.c 8
glamor_font.c 181 ++++++++++++++++++++
glamor_font.h 50 +++++
glamor_priv.h 26 ++
glamor_text.c 472 +++++++++++++++++++++++++++++++++++++++++++++++++++++
glamor_transform.c 2
8 files changed, 741 insertions(+), 5 deletions(-)
Let s Start At The Very Beginning
The results of optimizing text encouraged me to start at the top of
x11perf and see what progress I could make. In particular, looking at
the current Glamor code, I noticed that it did all of the vertex
transformation with the CPU. That makes no sense at all for any GPU
built in the last decade; they ve got massive numbers of transistors
dedicated to performing precisely this kind of operation. So, I
decided to see what I could do with PolyPoint.
PolyPoint is absolutely brutal on any GPU; you have to pass it two
coordinates for each pixel, and so the very best you can do is send it
32 bits, or precisely the same amount of data needed to actually draw
a pixel on the frame buffer. With this in mind, one expects that about
the best you can do compared with software is tie. Of course, the CPU
version is actually computing an address and clipping, but those are
all easily buried in the cost of actually storing a pixel.
In any case, the results of this little exercise are pretty close to a
tie the CPU draws 190,000,000 dots per second and the GPU draws
189,000,000 dots per second. Looking at the vertex and fragment
shaders generated by the compiler, it s clear that there s room for
improvement.
The fragment shader is simply pulling the constant pixel color from a
uniform and assigning it to the fragment color in this the simplest of
all possible shaders:
uniform vec4 color;
void main()
gl_FragColor = color;
;
This generates five instructions:
Native code for point fragment shader 7 (SIMD8 dispatch):
START B0
FB write target 0
0x00000000: mov(8) g113<1>F g2<0,1,0>F align1 WE_normal 1Q ;
0x00000010: mov(8) g114<1>F g2.1<0,1,0>F align1 WE_normal 1Q ;
0x00000020: mov(8) g115<1>F g2.2<0,1,0>F align1 WE_normal 1Q ;
0x00000030: mov(8) g116<1>F g2.3<0,1,0>F align1 WE_normal 1Q ;
0x00000040: sendc(8) null g113<8,8,1>F
render ( RT write, 0, 4, 12) mlen 4 rlen 0 align1 WE_normal 1Q EOT ;
END B0
As this pattern is actually pretty common, it turns out there s a
single instruction that can replace all four of the moves. That should
actually make a significant difference in the run time of this shader,
and this shader runs once for every single pixel.
The vertex shader has some similar optimization opportunities, but it
only runs once for every 8 pixels with the SIMD format flipped
around, the vertex shader can compute 8 vertices in parallel, so it
ends up executing 8 times less often. It s got some redundant moves,
which could be optimized by improving the copy propagation analysis
code in the compiler.
Of course, improving the compiler to make these cases run faster will
probably make a lot of other applications run faster too, so it s
probably worth doing at some point.
Again, the amount of code necessary to add this path was tiny:
Makefile.am 1
glamor.c 2
glamor_polyops.c 116 ++++++++++++++++++++++++++++++++++++++++++++++++++--
glamor_priv.h 8 +++
glamor_transform.c 118 +++++++++++++++++++++++++++++++++++++++++++++++++++++
glamor_transform.h 51 ++++++++++++++++++++++
6 files changed, 292 insertions(+), 4 deletions(-)
Discussion of Results
These two cases, text and points, are probably the hardest operations
to accelerate with a GPU and yet a small amount of OpenGL code was
able to meet or beat software easily. The advantage of this work over
traditional GPU 2D acceleration implementations should be pretty clear
this same code should work well on
any GPU which offers a reasonable
OpenGL implementation. That means everyone shares the benefits of this
code, and everyone can contribute to making 2D operations faster.
All of these measurements were actually done using Xephyr, which
offers a testing environment unlike any I ve ever had build
and test hardware acceleration code within a nested X server,
debugging it in a windowed environment on a single machine. Here s how
I m running it:
$ ./Xephyr -glamor :1 -schedMax 2000 -screen 1024x768 -retro
The one bit of magic here is the -schedMax 2000 flag, which causes
Xephyr to update the screen less often when applications are very busy
and serves to reduce the overhead of screen updates while running
x11perf.
Future Work
Having managed to accelerate 17 of the 392 operations in x11perf, it s
pretty clear that I could spend a bunch of time just stepping through
each of the remaining ones and working on them. Before doing that, we
want to try and work out some general principles about how to handle
core X fill styles. Moving all of the stipple and tile computation to
the GPU will help reduce the amount of code necessary to fill
rectangles and spans, along with improving performance, assuming the
above exercise generalizes to other primitives.
Getting and Testing the Code
Most of the changes here are from Eric s glamor-server branch:
git://people.freedesktop.org/~anholt/xserver glamor-server
The two patches shown above, along with a pair of simple clean up
patches that I ve written this week are available here:
git://people.freedesktop.org/~keithp/xserver glamor-server
Of course, as this now uses libepoxy, you ll need to fetch, build and
install that before trying to compile this X server.
Because you can try all of this out in Xephyr, it s easy to download
and build this X server and then run it right on top of your current
driver stack inside of X. I d really like to hear from people with
Radeon or nVidia hardware to know whether the code works, and how it
compares with fb on the same machine, which you get when you elide the
-glamor argument from the example Xephyr command line above.
(ARMv7 A15 dual core)
I ve covered how I set up my home LAVA lab and how I got two iMX.53 boards running, the next step was to sort out the LAVA server.
My initial setup relied on using my Thinkpad as the server. This was somewhat convenient as that s how I develop the code, but didn t help me run jobs like health checks because the laptop was usually somewhere other than on the home LAN. There had been a long-standing problem with using LAVA on ARM the uwsgi support didn t compile but the code being used was old. As the same code has been in Debian for a long time, I suspected that this was fixed with a newer version. So as part of migrating LAVA to django1.6, apache2.4, postgres9.3, django-tables2 0.13 and a few other updated dependencies, I have been using the version of uwsgi from Debian unstable for quite a while without problems on x86. A simple tweak to the LAVA setup scripts and a newer version of uwsgi became available and it builds just fine on armhf. (I haven t tried armel because Linaro is ARMv7 or newer and we don t test with ARMv5/ARMv6 boards, so there s no point in trying armel it s not likely to be an issue for this fix though.) (Only the current lava-deployment-tool methods need to actually compile uwsgi directly from upstream without any patches the packaging which will replace l-d-t uses precompiled binaries, just as any admin would expect.)
That initial install was on one of the iMX.53 boards, which ran as slowly as anyone who has used an iMX.53 would expect. I haven t run test jobs on that install, it had a hard enough time serving the frontend.
Clearly some more cores would be a good step and when an arndale became available for the home lab, the next step was to get a SATA drive onto the arndale, put Debian unstable on that and then deploy LAVA to the arndale using the Debian packages. As a bonus, these packages are built for django1.6 (using an unreleased branch).
Sounds simple enough. It works on iMX.53, it should work on an arndale. Well, it does but only after some unexpected faff.
arndale, u-boot and SATA
I hate u-boot every single time I ever touch it, the little I know about any other u-boot device is completely useless. Uniformity is a good thing infinite permutations are just rope to hang ourselves with cute embedded nonsense hacks and I m bored of that whole scene.
So, boot the arndale, interrupt the bootloader to get to the prompt. What s the first thing most people would want? help. Denied this particular u-boot has been compiled without the ability to display any help messages. u-boot has just sunk further into the depths of please can we have something else which is uniform across devices .
OK, well maybe I can get by without help. sata init unknown command sata . (Go away, do something else for a bit.)
The arndale isn t one of the newest boards in the ARMv7 world, LAVA has been using them for a while, so someone has probably done some work to fix this . nope. There s a workaround a nice detailed guide by Phil Endecott which details that the kernel needs to continue to be loaded from SD and then the SATA can be used for just the rootfs.
Mildly annoyed (I was hoping to be able to update the arndale kernel without the hassle of mounting the SD card at all), the rest was straight-forward.
Preparing the SATA and installing Debian
Due to the complexity of some of the transitions currently ongoing in Debian ahead of the Jessie release freeze, uwsgi is not in Debian testing right now so this install has to be Debian unstable. That s OK, I ve been running Debian unstable on all my machines for several years now usually updating ~5 times a week.
When I was recently talking about load-balancers, and automatically adding back-ends, not just removing bad ones that go offline, I obviously spent a while looking over some.
There are several dedicated load-balancers packaged for Debian GNU/Linux, including:
A new minor release 0.11.1 of
I switched to 
Go language has a strong type system unlike C, and some times this
will be head ache when we want to interact with C data types with Cgo
or just to convert a Go type to lets say byte slice. I recently faced
the same problem and after poking around things, I learned the Go
provides unsafe package which can be used to work around the Go's
type system.
"Given a Debian system with just the set of essential packages installed,
find a single package name that, together with its dependencies, will
increase the used diskspace of the system by as much as possible."
My entry: openarena.
Off to bed, now.
The following Python tree structure 
As a followup for my 
If that's what you're looking for, some variation on below may be of
use. In this approach, you build up a list of pages similar to the
paginator object page_range but with only the relevant pages
and the skip-markers identified.
I foolishly had a late night last night, for no particularly good reason, so I
slept in quite late this morning. I didn't start getting going until about 8am.
How decadent.
I'd forgotten to send Zoe to Kindergarten with her water bottle yesterday, and
forgotten to give it to Sarah when she picked her up yesterday, so I dropped it
and a fresh set of spare clothes out to Kindergarten after I had breakfast and
a shower. I managed to sneak in and out without Zoe seeing me.
I did some more work on my taxes before popping out for lunch to discuss my
main start up idea. I also renamed my old shelf company today.
I biked over to Kindergarten for pick up, and sure enough, Zoe was fast asleep.
It took quite a bit of work to rouse her. I was pretty sure she was just faking
it though. She moved pretty fast when I said I had a surprise at home for her.
We biked home, and I gave her the surprise (some Pop Rocks I'd been saving for
a special occasion) and headed out by car to a makeup Brazilian Jiu-Jitsu class
to make up for the one she missed on Friday.
It turned out that trying to do an afternoon class after Kindergarten and being
woken up from a nap wasn't the best idea. Zoe tired out pretty quickly, during
the warm up exercises, and was moderately uncooperative for the remainder of
the class (she kept going on strike).
I've never driven to or from BJJ classes until today, and I wasn't actually
sure how best to drive home, since we've always taken the Norman Park Greenway.
Zoe was asking for a "different way" anyway, so that worked out well. We winged
it through Coorparoo and the back of Norman Park and tacked onto the end of how
we'd normally cycle home.
Zoe had enough time to watch a little bit of TV before Sarah arrived to pick
her up.