Playing around with LLC controls on Sandybridge today.
What I wanted to know was how caching backbuffers affects application
performance. The reason for this question is that when the driver uses
page flipping to display a new frame, we force the new scanout buffer
to be uncached because the display hardware only scans out from main
memory. However, when we go back to using this buffer as the next
back-buffer, we don t turn the caching bits back on as that requires
writing GTT entries to change caching modes.
This means that swapping via page-flipping and swapping via copying
has a large change in main memory access patterns during rendering
page flipping applications use an uncached back buffer while copying
applications use a cached back buffer.
What effect does the back-buffer caching-mode have on application
performance?
The changes
In the kernel, I have three modes
1) Existing code. Set buffer objects to cache mode none when
preparing them as scanout buffers. The first time the application
flips to a back buffer, caching is disabled on that buffer forever
2) Flip back and forth. Set buffers to cache mode none when used
as scanout. When that stops, flip them back to cache mode llc.
This involves a whole lot of clflushing and GTT rewriting, obviously.
3) Flush to memory, but don t disable caching. In this mode, the
buffer gets flushed (from both CPU and GPU) before being used as a
scanout, but the cache mode isn t changed.
Benchmarking with glxgears
The first test was pretty simple I hacked up glxgears (yes,
glxgears is not a benchmark) to redraw the same frame multiple times
(including the clear) per swap to provide a scalable amount of
work per swap. No pixel operations, so this is just writing to the
back buffer. Thes numbers are the median of 5 runs, except for those
obviously limited by the refresh rate.
| Reps | Existing | Flip | Flush |
| 10 | 60 | 60 | 60 |
| 20 | 60 | 60 | 60 |
| 40 | 42.070 | 34.113 | 37.130 |
| 80 | 21.259 | 17.957 | 18.658 |
Note the linear scaling over reps (once you get below refresh). I
think this means that the swap overhead is in the noise, including
flushing and rewriting GTT entries. Given that my screen is (yuck)
1366x768 pixels, or just over 4MB, and my total LLC is 4MB, it s seems
like the performance changes here are likely caused by the rendering
ejecting every other object from the cache.
What s wrong with benchmarking with glxgears?
Of course, glxgears is
not a benchmark, and I realized while
collecting this data that, in particular, it never reads from the
backbuffer, and most pixels are only written once by the clear
operation at the start of the rendering code. So, it forces everything
else out of the cache at the start of each drawing cycle, but doesn t
stick anything useful into the cache.
Benchmarking nexuiz, a (more) real application
For the next test, I ran nexuiz demo1.dem in all three modes. Numbers
out of that weren t nearly as stable as glxgears, and I m afraid I
didn t write all of them down, but I got numbers between 45 and 50
frames per second
in all three modes. None significantly faster,
none significantly slower.
So, for these two tests, caching has no positive effect on overall
rendering performance. Obviously, I need to collect data from more
applications to see if the effect is general. I sure hope so, because
the alternative will be to find some heuristic to direct when to
enable caching.
The patch
This is on top of kernel version 3.5.
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index b0b676a..f69fba8 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1335,6 +1335,10 @@ int __must_check
i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
u32 alignment,
struct intel_ring_buffer *pipelined);
+
+void
+i915_gem_object_unpin_from_display_plane(struct drm_i915_gem_object *obj);
+
int i915_gem_attach_phys_object(struct drm_device *dev,
struct drm_i915_gem_object *obj,
int id,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 288d7b8..b606bd2 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2851,6 +2851,9 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
return 0;
+#define JUST_FLUSH 0
+#define FLIP_CACHING 0
+
/*
* Prepare buffer for display plane (scanout, cursors, etc).
* Can be called from an uninterruptible phase (modesetting) and allows
@@ -2883,7 +2886,11 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
* of uncaching, which would allow us to flush all the LLC-cached data
* with that bit in the PTE to main memory with just one PIPE_CONTROL.
*/
+#if JUST_FLUSH
+ ret = i915_gem_object_finish_gpu(obj);
+#else
ret = i915_gem_object_set_cache_level(obj, I915_CACHE_NONE);
+#endif
if (ret)
return ret;
@@ -2913,6 +2920,16 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
return 0;
+void
+i915_gem_object_unpin_from_display_plane(struct drm_i915_gem_object *obj)
+
+ i915_gem_object_unpin(obj);
+#if FLIP_CACHING
+ if (HAS_LLC(obj->base.dev))
+ i915_gem_object_set_cache_level(obj, I915_CACHE_LLC);
+#endif
+
+
int
i915_gem_object_finish_gpu(struct drm_i915_gem_object *obj)
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index a8538ac..76a2012 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -1821,7 +1821,7 @@ err_interruptible:
void intel_unpin_fb_obj(struct drm_i915_gem_object *obj)
i915_gem_object_unpin_fence(obj);
- i915_gem_object_unpin(obj);
+ i915_gem_object_unpin_from_display_plane(obj);
static int i9xx_update_plane(struct drm_crtc *crtc, struct drm_framebuffer *fb,
I'm forty pages into a battered copy of "Complicity" by Iain Banks. I picked
this copy up for the princely sum of 90p at Barter Books last weekend. For the
uninitiated, Barter Books is a grand old bookshop situated in a disused train
station in Alnwick, Northumberland. It has the dubious pleasure of being the
originator of the "keep calm and carry on" fad.
At the time that I bought it, I wasn't sure whether it was already in my
swollen reading pile, but 90p wasn't a great loss if so. I could leave it
in the book pile at my next hotel. (it was, and that's a sorry indictment of
the state of my reading pile.)
This gnarled paperback has character. There's a faded price sticker on the
back revealing a first-hand price in Australian Dollars. The book started its
post-sale life on the other side of the world. Where else has it been? I take
some small pleasure in planning the next step in its future.
To imagine this book with a long tail of readers having a free market value of
90p, or roughly the price of a posh chocolate bar, I'm struck by both wondering
how publishers and writers like Mr Banks make a living (and yet they do) but
also how the pricing of ebooks, completely lacking in this character I've
written about and non-transferable, seems over inflated to the end user.
I've been struggling to short list a selection of books to take on holiday.
Most of my friends have responded "buy a kindle", with a complete lack of
sympathy for my predicament. I'm borrowing one which means I can finally find
out whether I'll "take" to it. I'm concerned I'll not be able to get sucked
into a story in the same way I can with a paper book. I certainly won't rely on
just the kindle for my reading needs.
I don't want to prejudge my experiment too much but I suspect I won't be
convinced. Where I can see technology supporting my reading is in addition to
the paper book: particularly for non fiction, technical books, academic papers,
more graphical-oriented things (photo books, graphic novels). I do have a small
but growing collection of old, rare-in-print texts that I've only been able to
source as PDFs. I think something like the Google Nexus 7 would do the job for
these and fit into my life more readily than the kindle.
As I 
Last day of 
For the last months, I've been working on a metering project for
I did 
The brief outage was due to a scheduled move of the servers to a separate rack and subnet dedicated to our work with the Center for Information Assurance & Cybersecurity (ciac) at the University of Washington Bothell (uwb), and a11y.com
I am currently exercising the new (to us) equipment and hope to winnow the less than awesome equipment over the next quarter. I spent the last six months finding the best in breed of the surplussed DL385 and DL380 chassis we (work) were going to have recycled. The team and I were able to find enough equipment to bring up one of each with eight and six gigs of memory, respectively. These will make excellent hypervisors for provisioning embedded instances of Slackware, Fedora, RHEL, CentOS, Debian, FreeBSD, OpenSolaris, OpenIndiana, FreeDOS, etc.
When I initially configured this xen paravirt environment, I failed to plan for integration with libvirt, so I am now re-jiggering the software bridges so that they are managed using virt-manager. My experience doing this in the Eastsound lab has proved to be quite rewarding. Once network, storage and memory allocation are handled by libvirt, it will be trivial to automate the process of deploying, reclaiming, and managing these Telecommunications Resources efficiently and effectively using Perl.
The new equipment we racked over the last weekend all has hardware assisted virtualization and is capable of hosting operating systems which have not been patched to communicate with the host hypervisor. This provides a higher level of assurance that guests will receive the full amount of provisioned resources and reduces the likelihood of resource conflicts, should the cluster operations team mistakenly over-commit. It also makes it possible for these nodes to host Windows-based operating systems. Oh, and they will also host these not insignificant things called BIG-IP Local Traffic Manager Virtual Edition (VE) and BIG-IP 
After seeing 
With L szl B sz rm nyi we adopted 
I was invited onto NPR in Boston this week for a segment on 