So, I've been working on multitouch on and off the past few months (which
have included a solid ten weeks of holiday), but have finally posted the
third
patch series, which I think should be pretty close to final, to the list
today.
Touch events look kind of similar to standard Xi 2.1 events (DeviceMotionNotify
and friends), but the implementation of grabbing and event delivery is quite
different from the traditional X model that we all know and love.
Firstly, touch grabs are not strictly grabs as with other input events.
When a synchronous pointer or keyboard grab activates, it 'freezes' the device,
stopping delivery of all further events until the grabbing client has decided
what to do. The client can either pass on the event to the next client
(replay), opt to continue receiving all events with the device freezing again
after each event (synchronous mode), or continue receiving all events with
the device unfrozen (asynchronous). While this is happening, the grabbing
client is the only one receiving events, hence the name.
Touch grabs break both of these assumptions. Firstly, the device never
freezes: delivery of all events (pointer, keyboard, touch, whatever) continues
at full tilt, and a touch grab never affects pointer or keyboard events, with
the possible future exception of pointer events that are emulated from touch
events. Secondly, all clients with grabs, as well as the client with the first
selection we find<super>0</super>, receive the events. An 'owner' flag is set
(or unset) to indicate to the client that it does, or does not, own the touch
stream in question.
The owner principle is important: the theory is that all touch streams must be
considered as a whole (i.e. from init, through all the motion events, to the
touch stream ending), and cannot be separated into component parts. So a client
which does not own a touch stream can do gesture recognition or whathaveyou in
the background, but must not act upon it until it becomes the owner. We added
an extra request, XIAllowTouchEvents, to control ownership: clients can either
reject ownership, passing it on to the next in line, or assert their ownership
and permanently remove all other clients from the delivery list.
So it's somewhat different, but (hopefully) not without reason. The motivation
behind this was to keep latency as low as humanly possible, since people seem
to demand lower latencies from direct touch-based devices. Sticking to the
traditional grab model would have meant a flurry of round trips between all
the clients in the stack and the server, as well as maintaining huge buffers
in the server for every touch event, which each client would have to transfer
in turn.
This model is a compromise between my original design and Ubuntu's suggested
design; both of us had far more simple designs, but oriented towards different
usecases. The original Ubuntu usecase was for a global gesture recognition
system, which apps would only opt out of in rare cases. My primary usecase
was exactly the reverse: apps doing their own gesture recognition, with a
very small, if any, global system. Hopefully it's actually the best of both
worlds, rather than the worst possible intersection: I guess we'll find out. :)
At the moment, it's all largely theoretical, but some of the Nokia Qt guys are
working on a port and have provided really helpful feedback, and hopefully the
GTK+ multitouch branch, which was written against a much earlier and, er,
speculative, version of the spec, gets updated too.
Finally, some thanks: to both Nokia and my excellent employer Collabora for
sponsoring my part of this work; to Chase Douglas from Canonical for his
contributions and for not just giving us the finger and vanishing when we
rejected the X Gesture Extension, but hanging around and working through it;
and to Peter Hutterer, for spec work and review. There were also numerous
others involved earlier (including, but not limited to, Henrik Rydberg working
on mtdev and others) who deserve credit, so sorry if I forgot anyone.
If you want to get it and play around with it, the patches/repositories are
all laid out in my initial mail linked to earlier, as well as the follow-up
mail noting the evdev repository location. The
spec
has more details, and will hopefully be cleaned up to be a lot less
clunkily-written.
[0]: The delivery order is unchanged. For grabs, we start at the root window
and proceed down the window stack to the innermost child window, activating
grabs in that order. For events selected for with XISelectEvents, we start
at the innermost child window, work our way up towards the root window, and
stop as soon as we find one.