WM_TOUCH is totally bananas.

The iPad is one of the major platforms The Witness will appear on. Andy has been working on the iPad port and it is coming along. You can’t actually play the game yet, but we probably are close to that. The gameplay doesn’t know how to respond to a touch interface yet, though. It is important to me that this interface is very good, so I wanted to start designing for it now, to give me time to wrestle with difficult issues. Since the game runs already on Windows, it seemed natural to start designing the touch interface on Windows.

I am not sure that many people are going to use Windows for touch devices, at least in the Windows 8 timeframe. But at least some people will, so it’d be nice to have a high-quality touch interface built into the game on Windows.

With all this in mind, I started working on some Windows 8 touch hardware a couple of days ago. There is a message format that you can use in Windows 8 to get events from the touch hardware, WM_POINTER, but because I want the touch interface to work in Windows 7 as well, I wanted to use WM_TOUCH messages, which were introduced in Windows 7 and are supported in both operating systems.

It look me less than 20 minutes to get WM_POINTER working. It took many, many hours to get WM_TOUCH working, because WM_TOUCH is some kind of fully insane mess. None of the insanity seems to be documented, or discussed on the internet, so I am laying it out here in case anyone in the future stumbles into this same trap.

A WM_TOUCH message will never be returned by PeekMessage. Spy++ will tell you that WM_TOUCH is being sent to your application, but before the message is delivered, it gets stealthily converted into a mouse motion event. The type of this event will vary depending on whether you have raw input selected: if raw input, it will be WM_INPUT (0x00ff); if not, it will be WM_MOUSEMOTION (0×0200).

If your application handles these events, you may think it’s reasonable to complete the handling without calling DefWindowProc. If you do this, you will never see a WM_TOUCH, because something buried down in DefWindowProc stealthily dispatches a WM_TOUCH to your application’s windows when it sees a specially-marked WM_INPUT or WM_MOUSEMOVE.

Note that actually, you are supposed to always call DefWindowProc on WM_INPUT (for “cleanup” — I don’t know what this means and why they can’t just clean it up when you get the next message — but whatever); but WM_MOUSEMOVE has no such requirement, and if you believe you handled the event, it is natural *not* to call DefWindowProc. This will result in your never seeing WM_TOUCH. Strictly speaking our game had a bug, in that we were not calling DefWindowProc after WM_INPUT, but the bug was hard to find because it persisted when I tried turning off raw input, because we also were not calling DefWindowProc after WM_MOUSEMOVE (which as I mentioned is, in theory, totally fine.)

Important to note: WM_TOUCH gets dispatched to your windows without you dispatching it! You may think that you can log all messages after they come out of PeekMessage, and that you are in control of your entire message stream, but this is apparently not true. WM_TOUCH will just teleport itself into your window.

In addition to the mouse motion events, your application will get a bunch of other mouse events that are generated by the touch interface, to emulate mouse stuff for programs that don’t understand touch. There is no way to tell Windows you don’t want these events (unless you do Windows 8 stuff and respond to WM_POINTER; see below). According to Microsoft, the official way you are supposed to detect and ignore these messages is like this:

#define MOUSEEVENTF_FROMTOUCH 0xFF515700
if ((GetMessageExtraInfo() & MOUSEEVENTF_FROMTOUCH) 
        == MOUSEEVENTF_FROMTOUCH) { 
     // Skip message
}

I wish I were joking. It’s been this way for years; Casey wrote about this in 2008.

This mask works for events like WM_LBUTTONDOWN, but it does not work for the mouse motion events e.g. WM_INPUT. If you want to take input both from touch and the mouse, you need to disambiguate the real WM_INPUTs from the fake ones. Through a process of experimentation with message flags, I landed on this:


if ((GetMessageExtraInfo() & 0x82) == 0x82) { /* ignore event */ }

It seems to work for now, but who knows if that will mysteriously break in some future version of windows.

Nicely, if your application handles WM_POINTER messages, you will not get any of this mouse emulation stuff or WM_TOUCH, so all this ugliness disappears. If it is okay to make Windows 8 your minimum specification, I recommend using WM_POINTER and staying far away from WM_TOUCH.

This entry was posted in Development. Bookmark the permalink. Trackbacks are closed, but you can post a comment.

31 Comments

  1. Mellow
    Posted October 14, 2012 at 4:35 pm | Permalink

    Planned on iPad and Windows… Any word for Android people?

    • Jonathan Blow
      Posted October 14, 2012 at 5:46 pm | Permalink

      We will see about Android. I am not a fan of trying to build a robust experience across lots of heterogeneous unpredictable hardware-and-software profiles. But if it turns out that there are a couple of devices that are prominent and on which the game will run well, we’d probably release it.

      • Radiance
        Posted October 15, 2012 at 7:18 am | Permalink

        How about OUYA? I know it isn’t released yet, but:
        1. It’s has quite capable Tegra 3 GPU.
        2. It’s just a single device. No diversification.
        3. It’s a dedicated gaming platform, but without many quality games yet. Game as polished as The Witness will catch a lot of attention.
        4. It’s essentially just a usual android device, game will work on (powerful enough) android tablets with minimal changes.

        • Jonathan Blow
          Posted October 15, 2012 at 9:06 am | Permalink

          If and when the Ouya actually exists and people have bought it and are running games on it then we will look at it as a possibility.

          • Posted October 15, 2012 at 1:40 pm | Permalink

            Apportable has done a great job porting osmos to android. If you decide that it’s too much of a headache to do yourself, there are other options.

  2. Vincent Povirk
    Posted October 14, 2012 at 4:51 pm | Permalink

    Presumably DefWindowProc is dispatching the WM_TOUCH by calling SendMessage, which will always process the message synchronously. Sent messages are never returned from GetMessage or PeekMessage, just processed as soon as the window thread happens to be in a function that waits for messages (i.e. PeekMessage, GetMessage, or SendMessage). If the window happens to be in the same thread, SendMessage just calls the window proc.

    Because WM_INPUT is not a sent message, windows doesn’t know when you’re done processing it, and therefore it cannot know when it’s safe to free the information attached to it. In theory, you could save them up and process them in batches. I don’t know why anyone would do that, but it’s possible. (And, if MS were to free the handle on the next GetMessage call in the thread, anyone who tried to batch process them would likely have a similar reaction to the resulting confusing behavior.)

    Of course, given that you have to opt into WM_TOUCH, I don’t know why they didn’t just put it in the thread input queue and skip the corresponding WM_MOUSEMOVE/WM_INPUT messages. That would add a similar requirement to the one you have with WM_INPUT – since it would not be a posted message, and would have to contain more than 2 pointers worth of data, you’d need to call some other function to free the extra data.

    Incidentally, where is WM_POINTER documented? I’m interested in how they designed that differently, and I don’t see it anywhere on MSDN.

    • Jonathan Blow
      Posted October 14, 2012 at 5:45 pm | Permalink

      I *believe* WM_POINTER is documented in the Windows 8 SDK that was just released in mid-August. I haven’t verified this, though.

      The definitions do not exist even in the headers included with Visual Studio 2012. In order to reply to WM_POINTER messages I had to #define the constants myself, etc. (This doesn’t help so much when you want to access the structs!)

      • Ryan
        Posted October 16, 2012 at 8:53 am | Permalink

        This is correct. They are part of the Win8 SDK. Once that’s publically released (or if you have MSDN) – go grab it!

    • Posted October 14, 2012 at 6:14 pm | Permalink

      WM_POINTER is described in the following slides/talk.

      http://video.ch9.ms/build/2011/slides/APP-186T_Townsend.pptx
      http://channel9.msdn.com/Events/Build/BUILD2011/APP-186T

      The 50 min talk is very interesting if you enjoy the topic of input handling and touch.

      I’m a huge fan of the new WM_POINTER API, it’s the best thing coming from Microsoft that I’ve seen. I highly recommend watching the above video.

      • Vincent Povirk
        Posted October 15, 2012 at 10:25 pm | Permalink

        Very interesting video, thanks for the link.

        I’ve found what appears to be the WM_POINTER documentation here: http://msdn.microsoft.com/en-us/library/windows/desktop/hh454903%28v=vs.85%29.aspx
        No usage guide there yet, but the video makes a good introduction (even though it does not talk about WM_POINTER much at all, and instead focuses on WinRT api’s, they are apparently designed to be analogous).

        Apparently, there is not a message called WM_POINTER but a variety of message types, which is probably why I did not find this before. The extra information is retrieved by calling a function, and only information for the last returned pointer message from a specific device (and, if the message was coalesced, any events that happened between messages) is available. It does seem more intuitive.

  3. Ricardo
    Posted October 14, 2012 at 4:56 pm | Permalink

    I’m amazed at how Apple makes things simple for developers, and how Microsoft screws their APIs time after time.

    • Ryan
      Posted October 16, 2012 at 8:49 am | Permalink

      This comment amuses me. Do you ever develop for Apple products? Time after time? You don’t know what you’re talking about. Just another Apple fanboy.

      The point of this article was that WM_TOUCH was poorly done and the author kindly explained some hurdles he went over. I have used WM_POINT* messages extensively and they are fantastic for custom touch control.

  4. Daniel R. Maciel
    Posted October 14, 2012 at 5:17 pm | Permalink

    I guess not many people care about WM_TOUCH. It is always cool to have news about The Witness though.
    BTW, I’m ready to thank you for your good work with my monies, is the game out anytime soon? :D

  5. WC
    Posted October 14, 2012 at 5:38 pm | Permalink

    Oh joy, magic numbers!

    Interesting read. Thanks for posting this for others to learn from. I hope that MS fixes this insanity before I have to deal with it, but somehow I doubt that’ll be the case. ;)

  6. Posted October 14, 2012 at 6:08 pm | Permalink

    The Win32 message loop is all kinds of crazy.

    I really dislike DefWindowProc in particular. It is undocumented, you don’t have access to its source code, and worst of all, it completely blocks your main (where typically your OpenGL rendering happens) thread while the user moves or resizes the window. You have no way around your main thread blocked, unless you’re willing to avoid using DefWindowProc by reimplementing all default windows behaviours (moving, resizing, minimizing, etc.), which is a lot of work and will invariably have your app acting differently from the standard Windows experience.

    That said, I still think you could’ve debugged this problem faster if you had started with sample code where WM_TOUCH was being received, and changed its message loop one step at a time to be like yours and checked which change breaks WM_TOUCH being received. Or maybe it’d be slower that way, I dunno.

  7. knight666
    Posted October 15, 2012 at 1:22 am | Permalink

    I’ve worked on a project using WM_TOUCH events extensively and let me tell you: you’re doing it wrong.

    You complain that WM_TOUCH is being converted to WM_INPUT and WM_MOUSEMOTION. Obviously this is to support applications that are unaware of any WM_TOUCH events being generated. However, there’s a very simple way to work around this: register your application to receive touch!

    Call “RegisterTouchWindow” (http://msdn.microsoft.com/en-us/library/windows/desktop/dd317326(v=vs.85).aspx) at the start of your application and you will receive WM_TOUCH messages without them being converted to mouse events.

    Check out the development branch of GLFW that includes my patches: https://github.com/elmindreda/glfw/tree/touch. See the functions “_glfwPlatformSetTouchInput” and look for “WM_TOUCH” in “win32_window.c”.

    • Jonathan Blow
      Posted October 15, 2012 at 9:17 am | Permalink

      If I were you I would not be so sure of myself (seemingly to the point of rudeness as you are here).

      Of course the game calls RegisterTouchWindow, because if you don’t call this, you don’t receive any touch events at all. (To verify this for the 20th time, I just tried taking out the RegisterTouchWindow call again, before writing this reply). When you do receive these touch events, it works as I have described: through mouse events. There is no apparent way not to receive the mouse events, as far as I can see.

      For all I know this may be behavior specific to Windows 8. I haven’t tried it on Windows 7 yet. What I have described is merely the extremely consistent and reproducible behavior of the message system on this one Windows 8 machine that I have.

      • Andrew Lauritzen
        Posted October 15, 2012 at 1:59 pm | Permalink

        It works that way on Windows 7 too… you still receive the union of “pure” mouse events and touch-based ones, and it’s quite annoying. Will take a look at WM_POINTER, since in my case I’m not too concerned about the small number of Windows 7 tablet/touch devices out there.

      • Ryan
        Posted October 16, 2012 at 8:52 am | Permalink

        From what I remember, WM_TOUCH operates a bit differently on Win8 than it did on Win7. I believe Windows cut some of the support and my project saw some interesting regressions between the two versions of the OS. I apologize though, I can’t remember exactly what it was.

  8. Posted October 15, 2012 at 4:54 am | Permalink

    Seeing that you released Braid on Linux, not for sales but for ethical reasons, I had always imagined you would embrace Android, especially over iOS. Maybe I’m a bit prejudiced, either way I’m looking forward to seeing it on Windows, I read your previous rant about developing for consoles, but I’d love for it to be on PS3 too.
    Whatever happens, I look forward to this next piece of genius, and the subsequent lectures to follow. I hope there isn’t too much pressure from hype and from Braid’s success.

  9. Sami
    Posted October 15, 2012 at 7:07 am | Permalink

    Despite my coding experience going no further than ActionScript 3, I still read the whole post. It was pretty interesting, and if I ever need to develop for touchscreen Windows and use WM_TOUCH I’ll know where to look for some help.

  10. alasta
    Posted October 15, 2012 at 4:13 pm | Permalink

    How can you use touch screen to navigate a 3D space? I can only see it being done like ios Rage HD where it’s on rails. Touch buttons etc. are really annoying.

    • Jonathan Blow
      Posted October 15, 2012 at 7:12 pm | Permalink

      I wouldn’t do it for an action game, but for a slower-paced game it isn’t a problem. Try downloading Epic Citadel / etc.

  11. justin
    Posted October 17, 2012 at 8:30 pm | Permalink

    I have nothing to say about this touch stuff… But just to expand on the architecture stuff: This applies to game design

    http://youtu.be/Q1ZeXnmDZMQ

    Special thanks to Instigative Catwoman for sharing…
    Capcha: Illustrating this

  12. Posted October 21, 2012 at 11:30 am | Permalink

    was just wondering if you would include the xbox controller support on the surface running win 8 that was just announced a lil while ago?

    • Jonathan Blow
      Posted October 22, 2012 at 10:53 am | Permalink

      We would explicitly have to turn off 360 controller support, and I can’t think of any reason we would do that (unless there were some kind of unforeseen technical problem.)

  13. Braidlover
    Posted November 3, 2012 at 2:41 pm | Permalink

    If you are developing for win8, why not use RT Apis?
    They have built in touch support that is a lot better than WM_TOUCH messages…. And a lot cleaner as well.

  14. Braidlover
    Posted November 3, 2012 at 2:43 pm | Permalink

    I see that you want to support Win7 as well… Yeah never mind :)

  15. Posted March 20, 2013 at 1:34 pm | Permalink

    So, Jonathan, how did you ever then manage to catch WM_TOUCH events?? Many, many thanks for sharing this experience. Wow.

  16. Posted May 14, 2013 at 2:45 pm | Permalink

    I’ve just been through this horror myself, ending up with the same approach. I wish I’d found this blog post earlier.

    I had one further problem, which was that I want to be able to use pen input as well as mouse and touch. Pen events come through to WM_MOUSEMOVE etc marked with MOUSEEVENTF_FROMTOUCH, *but* they don’t get a WM_TOUCH event. You have to separate “from touch” events into “from pen” and “from finger”, and process pen events like a mouse. Touches from a finger have bit 0×80 set in GetMessageExtraInfo, pen messages haven’t. The lower 7 bits are the touch device id. The 0×82 you found from experimenting is presumably 0×80 plus a device id, and may not work on all machines.

    It was only by working this out through trial and error that I had enough information to find this MSDN article confirming it: http://msdn.microsoft.com/en-us/library/windows/desktop/ms703320%28v=vs.85%29.aspx

Post a Comment

Your email is never published nor shared.

You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>