The iPad is one of the major platforms The Witness will appear on. Andy has been working on the iPad port and it is coming along. You can't actually play the game yet, but we probably are close to that. The gameplay doesn't know how to respond to a touch interface yet, though. It is important to me that this interface is very good, so I wanted to start designing for it now, to give me time to wrestle with difficult issues. Since the game runs already on Windows, it seemed natural to start designing the touch interface on Windows.
I am not sure that many people are going to use Windows for touch devices, at least in the Windows 8 timeframe. But at least some people will, so it'd be nice to have a high-quality touch interface built into the game on Windows.
With all this in mind, I started working on some Windows 8 touch hardware a couple of days ago. There is a message format that you can use in Windows 8 to get events from the touch hardware, WM_POINTER, but because I want the touch interface to work in Windows 7 as well, I wanted to use WM_TOUCH messages, which were introduced in Windows 7 and are supported in both operating systems.
It look me less than 20 minutes to get WM_POINTER working. It took many, many hours to get WM_TOUCH working, because WM_TOUCH is some kind of fully insane mess. None of the insanity seems to be documented, or discussed on the internet, so I am laying it out here in case anyone in the future stumbles into this same trap.
A WM_TOUCH message will never be returned by PeekMessage. Spy++ will tell you that WM_TOUCH is being sent to your application, but before the message is delivered, it gets stealthily converted into a mouse motion event. The type of this event will vary depending on whether you have raw input selected: if raw input, it will be WM_INPUT (0x00ff); if not, it will be WM_MOUSEMOTION (0x0200).
If your application handles these events, you may think it's reasonable to complete the handling without calling DefWindowProc. If you do this, you will never see a WM_TOUCH, because something buried down in DefWindowProc stealthily dispatches a WM_TOUCH to your application's windows when it sees a specially-marked WM_INPUT or WM_MOUSEMOVE.
Note that actually, you are supposed to always call DefWindowProc on WM_INPUT (for "cleanup" -- I don't know what this means and why they can't just clean it up when you get the next message -- but whatever); but WM_MOUSEMOVE has no such requirement, and if you believe you handled the event, it is natural *not* to call DefWindowProc. This will result in your never seeing WM_TOUCH. Strictly speaking our game had a bug, in that we were not calling DefWindowProc after WM_INPUT, but the bug was hard to find because it persisted when I tried turning off raw input, because we also were not calling DefWindowProc after WM_MOUSEMOVE (which as I mentioned is, in theory, totally fine.)
Important to note: WM_TOUCH gets dispatched to your windows without you dispatching it! You may think that you can log all messages after they come out of PeekMessage, and that you are in control of your entire message stream, but this is apparently not true. WM_TOUCH will just teleport itself into your window.
In addition to the mouse motion events, your application will get a bunch of other mouse events that are generated by the touch interface, to emulate mouse stuff for programs that don't understand touch. There is no way to tell Windows you don't want these events (unless you do Windows 8 stuff and respond to WM_POINTER; see below). According to Microsoft, the official way you are supposed to detect and ignore these messages is like this:
#define MOUSEEVENTF_FROMTOUCH 0xFF515700
if ((GetMessageExtraInfo() & MOUSEEVENTF_FROMTOUCH)
== MOUSEEVENTF_FROMTOUCH) {
// Skip message
}
I wish I were joking. It's been this way for years; Casey wrote about this in 2008.
This mask works for events like WM_LBUTTONDOWN, but it does not work for the mouse motion events e.g. WM_INPUT. If you want to take input both from touch and the mouse, you need to disambiguate the real WM_INPUTs from the fake ones. Through a process of experimentation with message flags, I landed on this:
if ((GetMessageExtraInfo() & 0x82) == 0x82) { /* ignore event */ }
It seems to work for now, but who knows if that will mysteriously break in some future version of windows.
Nicely, if your application handles WM_POINTER messages, you will not get any of this mouse emulation stuff or WM_TOUCH, so all this ugliness disappears. If it is okay to make Windows 8 your minimum specification, I recommend using WM_POINTER and staying far away from WM_TOUCH.
Planned on iPad and Windows… Any word for Android people?
We will see about Android. I am not a fan of trying to build a robust experience across lots of heterogeneous unpredictable hardware-and-software profiles. But if it turns out that there are a couple of devices that are prominent and on which the game will run well, we’d probably release it.
How about OUYA? I know it isn’t released yet, but:
1. It’s has quite capable Tegra 3 GPU.
2. It’s just a single device. No diversification.
3. It’s a dedicated gaming platform, but without many quality games yet. Game as polished as The Witness will catch a lot of attention.
4. It’s essentially just a usual android device, game will work on (powerful enough) android tablets with minimal changes.
If and when the Ouya actually exists and people have bought it and are running games on it then we will look at it as a possibility.
Apportable has done a great job porting osmos to android. If you decide that it’s too much of a headache to do yourself, there are other options.
Presumably DefWindowProc is dispatching the WM_TOUCH by calling SendMessage, which will always process the message synchronously. Sent messages are never returned from GetMessage or PeekMessage, just processed as soon as the window thread happens to be in a function that waits for messages (i.e. PeekMessage, GetMessage, or SendMessage). If the window happens to be in the same thread, SendMessage just calls the window proc.
Because WM_INPUT is not a sent message, windows doesn’t know when you’re done processing it, and therefore it cannot know when it’s safe to free the information attached to it. In theory, you could save them up and process them in batches. I don’t know why anyone would do that, but it’s possible. (And, if MS were to free the handle on the next GetMessage call in the thread, anyone who tried to batch process them would likely have a similar reaction to the resulting confusing behavior.)
Of course, given that you have to opt into WM_TOUCH, I don’t know why they didn’t just put it in the thread input queue and skip the corresponding WM_MOUSEMOVE/WM_INPUT messages. That would add a similar requirement to the one you have with WM_INPUT – since it would not be a posted message, and would have to contain more than 2 pointers worth of data, you’d need to call some other function to free the extra data.
Incidentally, where is WM_POINTER documented? I’m interested in how they designed that differently, and I don’t see it anywhere on MSDN.
I *believe* WM_POINTER is documented in the Windows 8 SDK that was just released in mid-August. I haven’t verified this, though.
The definitions do not exist even in the headers included with Visual Studio 2012. In order to reply to WM_POINTER messages I had to #define the constants myself, etc. (This doesn’t help so much when you want to access the structs!)
This is correct. They are part of the Win8 SDK. Once that’s publically released (or if you have MSDN) – go grab it!
WM_POINTER is described in the following slides/talk.
http://video.ch9.ms/build/2011/slides/APP-186T_Townsend.pptx
http://channel9.msdn.com/Events/Build/BUILD2011/APP-186T
The 50 min talk is very interesting if you enjoy the topic of input handling and touch.
I’m a huge fan of the new WM_POINTER API, it’s the best thing coming from Microsoft that I’ve seen. I highly recommend watching the above video.
Very interesting video, thanks for the link.
I’ve found what appears to be the WM_POINTER documentation here: http://msdn.microsoft.com/en-us/library/windows/desktop/hh454903%28v=vs.85%29.aspx
No usage guide there yet, but the video makes a good introduction (even though it does not talk about WM_POINTER much at all, and instead focuses on WinRT api’s, they are apparently designed to be analogous).
Apparently, there is not a message called WM_POINTER but a variety of message types, which is probably why I did not find this before. The extra information is retrieved by calling a function, and only information for the last returned pointer message from a specific device (and, if the message was coalesced, any events that happened between messages) is available. It does seem more intuitive.
I’m amazed at how Apple makes things simple for developers, and how Microsoft screws their APIs time after time.
This comment amuses me. Do you ever develop for Apple products? Time after time? You don’t know what you’re talking about. Just another Apple fanboy.
The point of this article was that WM_TOUCH was poorly done and the author kindly explained some hurdles he went over. I have used WM_POINT* messages extensively and they are fantastic for custom touch control.
I guess not many people care about WM_TOUCH. It is always cool to have news about The Witness though.
BTW, I’m ready to thank you for your good work with my monies, is the game out anytime soon? :D
Oh joy, magic numbers!
Interesting read. Thanks for posting this for others to learn from. I hope that MS fixes this insanity before I have to deal with it, but somehow I doubt that’ll be the case. ;)
The Win32 message loop is all kinds of crazy.
I really dislike DefWindowProc in particular. It is undocumented, you don’t have access to its source code, and worst of all, it completely blocks your main (where typically your OpenGL rendering happens) thread while the user moves or resizes the window. You have no way around your main thread blocked, unless you’re willing to avoid using DefWindowProc by reimplementing all default windows behaviours (moving, resizing, minimizing, etc.), which is a lot of work and will invariably have your app acting differently from the standard Windows experience.
That said, I still think you could’ve debugged this problem faster if you had started with sample code where WM_TOUCH was being received, and changed its message loop one step at a time to be like yours and checked which change breaks WM_TOUCH being received. Or maybe it’d be slower that way, I dunno.
I’ve worked on a project using WM_TOUCH events extensively and let me tell you: you’re doing it wrong.
You complain that WM_TOUCH is being converted to WM_INPUT and WM_MOUSEMOTION. Obviously this is to support applications that are unaware of any WM_TOUCH events being generated. However, there’s a very simple way to work around this: register your application to receive touch!
Call “RegisterTouchWindow” (http://msdn.microsoft.com/en-us/library/windows/desktop/dd317326(v=vs.85).aspx) at the start of your application and you will receive WM_TOUCH messages without them being converted to mouse events.
Check out the development branch of GLFW that includes my patches: https://github.com/elmindreda/glfw/tree/touch. See the functions “_glfwPlatformSetTouchInput” and look for “WM_TOUCH” in “win32_window.c”.
If I were you I would not be so sure of myself (seemingly to the point of rudeness as you are here).
Of course the game calls RegisterTouchWindow, because if you don’t call this, you don’t receive any touch events at all. (To verify this for the 20th time, I just tried taking out the RegisterTouchWindow call again, before writing this reply). When you do receive these touch events, it works as I have described: through mouse events. There is no apparent way not to receive the mouse events, as far as I can see.
For all I know this may be behavior specific to Windows 8. I haven’t tried it on Windows 7 yet. What I have described is merely the extremely consistent and reproducible behavior of the message system on this one Windows 8 machine that I have.
It works that way on Windows 7 too… you still receive the union of “pure” mouse events and touch-based ones, and it’s quite annoying. Will take a look at WM_POINTER, since in my case I’m not too concerned about the small number of Windows 7 tablet/touch devices out there.
From what I remember, WM_TOUCH operates a bit differently on Win8 than it did on Win7. I believe Windows cut some of the support and my project saw some interesting regressions between the two versions of the OS. I apologize though, I can’t remember exactly what it was.
Seeing that you released Braid on Linux, not for sales but for ethical reasons, I had always imagined you would embrace Android, especially over iOS. Maybe I’m a bit prejudiced, either way I’m looking forward to seeing it on Windows, I read your previous rant about developing for consoles, but I’d love for it to be on PS3 too.
Whatever happens, I look forward to this next piece of genius, and the subsequent lectures to follow. I hope there isn’t too much pressure from hype and from Braid’s success.
Despite my coding experience going no further than ActionScript 3, I still read the whole post. It was pretty interesting, and if I ever need to develop for touchscreen Windows and use WM_TOUCH I’ll know where to look for some help.
How can you use touch screen to navigate a 3D space? I can only see it being done like ios Rage HD where it’s on rails. Touch buttons etc. are really annoying.
I wouldn’t do it for an action game, but for a slower-paced game it isn’t a problem. Try downloading Epic Citadel / etc.
I have nothing to say about this touch stuff… But just to expand on the architecture stuff: This applies to game design
http://youtu.be/Q1ZeXnmDZMQ
Special thanks to Instigative Catwoman for sharing…
Capcha: Illustrating this
was just wondering if you would include the xbox controller support on the surface running win 8 that was just announced a lil while ago?
We would explicitly have to turn off 360 controller support, and I can’t think of any reason we would do that (unless there were some kind of unforeseen technical problem.)
thank you
If you are developing for win8, why not use RT Apis?
They have built in touch support that is a lot better than WM_TOUCH messages…. And a lot cleaner as well.
I see that you want to support Win7 as well… Yeah never mind :)
So, Jonathan, how did you ever then manage to catch WM_TOUCH events?? Many, many thanks for sharing this experience. Wow.
I’ve just been through this horror myself, ending up with the same approach. I wish I’d found this blog post earlier.
I had one further problem, which was that I want to be able to use pen input as well as mouse and touch. Pen events come through to WM_MOUSEMOVE etc marked with MOUSEEVENTF_FROMTOUCH, *but* they don’t get a WM_TOUCH event. You have to separate “from touch” events into “from pen” and “from finger”, and process pen events like a mouse. Touches from a finger have bit 0x80 set in GetMessageExtraInfo, pen messages haven’t. The lower 7 bits are the touch device id. The 0x82 you found from experimenting is presumably 0x80 plus a device id, and may not work on all machines.
It was only by working this out through trial and error that I had enough information to find this MSDN article confirming it: http://msdn.microsoft.com/en-us/library/windows/desktop/ms703320%28v=vs.85%29.aspx
I can’t seem to get touch controls to work on my Razer Edge Pro Windows tablet. On the first screen I can swipe and manipulate the camera slightly but the arrow icons on the bottom right do not respond to touch. Any fix for this?
Touch controls are not enabled yet. We plan to turn them on before too long but for now it’s mouse/keyboard or gamepad only.
Thanks for the speedy response, can’t wait to play it using the touchscreen it seems like such a natural fit.
Any update on this? I’ve made the increasingly questionable decision to transition from an aging but reasonably well spec-ed desktop rig to a 5 year old refurbished Windows 7 tablet as my sole pc for impending travel. I’ll have a little handheld trackball and a 360 pad with me anyway, so it certainly won’t stop me from playing the game, but I was looking forward to the tactile experience of tracing the puzzles with my finger.
We haven’t finished touch support yet for any platform because it requires good player pathfinding / etc through the world, and we are working on that.
Still patiently waiting for this! Can’t believe I’ve been able to hold off playing, hopefully touch support coming soon.
Is it safe to assume PC touch support won’t be available until an iOS release ?
Jonathan you’re the man!
Just spent hours trying to figure out why my Python / Kivy app on the Surface Pro 4 was being random and recognizing some, not other touch interactions.
Your article pointed me in the right direction for the fix. The app was trying to respond to both mouse and wm_touch messages. Disabling the app’s attempts to handle wm_touch itself allowed the touch events to be converted into mouse events and be processed properly by the app.
Thanks – extremely helpful :)
Still patiently waiting almost a year later (lol). Maybe I missed it but has an update been released to enable touch controls yet?
So now that the Witness is available on iOS is it safe to assume touch support is added on PC version? Bought the game on Day 1. Still haven’t played it and would hate to buy it again on iPad
?
Thank you, this article was helpful along with the comment and link by James Brown. That link currently redirects to the Microsoft “System Events and Mouse Messages” documentation.
The Microsoft documentation defines the following three macros which can be used to ignore the mouse events which Window generates from pen devices.
#define MI_WP_SIGNATURE 0xFF515700
#define SIGNATURE_MASK 0xFFFFFF00
#define IsPenEvent(dw) (((dw) & SIGNATURE_MASK) == MI_WP_SIGNATURE)
The IsPenEvent macro will also identify mouse events Windows generates from touch devices. Using this information we were able to help a customer get touch events working in their application which uses our 2D dynamic graphics engine.
Thanks again!