We haven't done one of these in a while. Here's what the island looks like currently:
The current puzzle count is 497. (This is lower than last time; last time was an overcount, as we had some temporary duplicates in the world that were being counted mistakenly.)
In The Witness, when progress in your game is saved, we also save a thumbnail that shows you where you were. This makes it easier to load a particular game later if you had multiple saves. Here is our current, very unpolished load screen:
Right now, we save the thumbnail and the game data as separate files, but the names are mostly the same; so if you go into your game data folder in Windows you can see them living there very plainly:
(It might be interesting to do it like Spore and embed the save data in the image file, so that there's only one file per save. Maybe we'll do this, though it's not a no-brainer; right now the image file is usually bigger than the save data, so this would involve tying a relatively small amount of crucial data to a relatively large amount of cosmetic data. Maybe that doesn't matter, though.)
As I mentioned in the previous post, we are starting to get the game out to a limited number of players. It's been a long time since I've had anyone new play through the game as a whole, so I really have no idea how it plays now, and it's important to find out. However, it's not so easy, because if you sit behind someone and watch them play a game, they are going to play it differently than if you weren't there. You invalidate your own playtest data.
So, I wanted to set up the game to record some minimal playtest data, that would be easy for me to browse through later on, and that would get recorded without me being there to directly watch the play session.
One of the first approaches that programmers often consider is an input recording method. You just journal a stream of all the player's inputs, and you can play that back deterministically to reproduce the entire game session. I have never liked this method. Firstly, you need to play back using exactly the same version of the game executable that the player used, which is a pain; secondly, this kind of playback code is brittle and breaks often. An uninitialized memory read bug can render your playback data useless! (Of course, one reason you might want playback data is to track down an uninitialized memory read bug... so... there's a problem here.)
For these reasons I didn't want to use input journaling. It seemed to me that our saved games are small enough, and the thumbnails are nice enough, that together they almost solve the problem. All I need to do is save a series of snapshots / game states over time, without overwriting previous saves:
Whereas the earlier folder contained a collection of different play sessions, this one contains one play session at different moments in time. Every time the game made an autosave for this play session (overwriting the old one in the normal way), it also also copied the autosave here. The date and time that are in each filename just indicate when that game was started; the number at the end tells you which snapshot this is.
There's one immediate, very nice property of this: I don't even need to run the game in order to get a general idea of how someone's playtest went. Once they send me the files, I can just click on the first one in Windows to invoke the image viewer, and then click the forward/backward buttons in the image viewer to animate my way through the game.
But, if I have questions about what was going on, I can also dig into the save file that corresponds to any given image. I made a mode that lets me browse these saves in-game, load up any particular one into the current world state, and play from there. I invoke it by running a console command:
This loads all the saved games and thumbnails and pops them into a scrolling list at the top of the screen, in chronological order:
I just scroll to the one I want:
... then press Enter to load that game state:
It's very fast to use and it only took a few hours to program.
This whole system is also pretty cool because it's much more robust than the input-recording method. If I want to look into some detail that is super-nitpicky, I may need to use the same build of the game executable that the player used, but this will almost never be necessary. Usually I won't even need to use the in-game browser -- as I mentioned, I can just click through the thumbnails in Windows to get a basic sense of what's going on.
The fact that we only have discrete snapshots is sort of a side-bonus: because we only snapshot every minute or so, the data is not so creepy and spy-like; players can relax while they are playtesting and know that we won't later be obsessing over their every move.
Last Wednesday night, just before Thanksgiving, we shipped our first extremely closed beta over Steam to a few limited testers.
Today we shipped an update, still to the same limited group. We have some initial tech issues to work out before we go wider (game packaging issues, graphics card incompatibilities, and the like).
Here's a desktop screenshot from one of the testers, with someone chiding him about what he's playing:
(The Witness window in this shot is artificially resized to make room on the desktop; by default we launch at 1280x720.)
I know that the impulse is to think "oh wow they are almost done!" Don't think that, it's not the case. We still have a gigantic amount of work to do and it'll be a while before the game is done.
Random Trivia Q&A
Q: How many footstep sound effects are in The Witness?
A: 1,119 so far. They sound really good! We will probably be in the Guinness Book of World Records as the game with the most footstep sounds...
I really like the scope(exit) statements in D. They allow you to keep the cleanup code next to the initialization code making it much easier to maintain and understand. I wish we had them in C++.
Turns out this can be implemented easily using some of the new C++ features. I'm not a big fan of C++11; for the most part I think it adds even more complexity to an already complex language. However, I have to admit that some of the features can be useful, when used with care and moderation.
I think this is one of such cases. If you can get past the weird syntax, lambdas can be very powerful. What I find most useful is to be able to define local functions (closures) that can access the local variables of the enclosing scope.
In our case, what we want to do is to define some code that is executed at the end of the scope to cleanup objects allocated in the current scope. This is easily achieved allocating an object on the stack that invokes a lambda function that wraps our custom cleanup code:
#include <functional>
struct ScopeExit {
ScopeExit(std::function f) : f(f) {}
~ScopeExit() { f(); }
std::function f;
};
#define STRING_JOIN2(arg1, arg2) DO_STRING_JOIN2(arg1, arg2)
#define DO_STRING_JOIN2(arg1, arg2) arg1 ## arg2
#define SCOPE_EXIT(code) \
ScopeExit STRING_JOIN2(scope_exit_, __LINE__)([=](){code;})
Then you can write code like:
int * ip = new int[16];
SCOPE_EXIT(delete [] ip);
FILE * fp = fopen("test.out", "wb");
SCOPE_EXIT(fclose(fp));
I know you can go to greater lengths to mimic the D syntax, and that Boost also provides similar functionality, but this does all we need in less than 10 lines of code.
Sadly we cannot really use this on The Witness, because I'm not sure we are yet ready to drop support for platforms and compilers that do not implement the required C++11 functionality.
Update:
As mentioned by Daniel in the comments, you don't really want to use std::function in this case. A quick look at the assembly output confirms that the generated code is horrible. It makes a lot more sense to wrap the closure explicitly and rely on type inference to allocate the wrapper object on the stack:
template <typename F>
struct ScopeExit {
ScopeExit(F f) : f(f) {}
~ScopeExit() { f(); }
F f;
};
template <typename F>
ScopeExit<F> MakeScopeExit(F f) {
return ScopeExit<F>(f);
};
#define SCOPE_EXIT(code) \
auto STRING_JOIN2(scope_exit_, __LINE__) = MakeScopeExit([=](){code;})
... when even your barest rough drafts seem like they'd be interesting to explore.
Current puzzle count: 528.
As mentioned in the previous post, we shrank the island a bit. Here is what things look like now.
They may not look tremendously different from this off-island vantage point, but when you are down there moving around, the difference is substantial. I feel a lot better about the layout now.
One of my primary design goals, all through development, has been to keep the island spatially small, so that there isn't a lot of wandering through empty or pointless spaces. The idea is to take the best parts of an open-world game, involving choice, exploration and discovery, but compressing these into a maximally dense experience.
That has been going well, but there's an interesting phenomenon where, as we add more detail to the world, it starts to feel a lot bigger. When something is a mostly-featureless plain it is easy to feel like it's small, but once you are passing tiny rocks and plants that are set up very carefully, your brain has more scale reference. So there have been a couple of times during development where we stopped and said "Hey wait a minute, things are just too big."
Last week that happened again; we were looking at two different parts of the island on the same day, and both of them seemed much bigger than we needed. So, we decided to go back to the overall island plan and try to compress it further. We had a discussion about this on Thursday, then the architects talked it over; you see the result of those discussions in the image above. These maps are a handy guide to helping us figure out what we can cut and how we can scoot existing areas around to pack them a little better, and to better-define the areas between them.
This is a little painful, since we almost had the island back into playable shape, and this knocks it back out of playable shape for a little longer. But my hope is that we can just make a mess and then clean it up very quickly.
In other news, there had been a puzzle that I needed to design for two years -- in the sense that I knew I needed to build the puzzle in order for a certain area to be complete, and I knew what that puzzle vaguely had to be about, but didn't have any good ideas on how to build it. Last week I had a breakthrough on that and built it very quickly; it's now one of the best puzzles in the game. So things are coming along well!
For the past few weeks Andy has been working on the iPad port of the game, and we now have the engine running on that device:
The colors are all funny because we haven't written the shaders for the iPad yet, so what you are seeing is the surface normals of the objects in the scene.
We haven't written the shaders yet because they need to be OpenGL ES shaders, and the shaders we have right now are HLSL shaders all written for medium-to-high-end PCs. As the first part of the porting work, Andy made the engine work on OpenGL on the PC, and for that we converted our HLSL shaders to GLSL automatically. But the jump to ES on an embedded device is big enough that it doesn't make sense to try and use the same shaders. Instead we'll have to have some simpler shaders that we substitute.
As you can see from my previous post, I have been working on touch controls (not on the iPad, but on a touchscreen Windows PC). A very rough first cut of that is now working. It's nice to play the game with touch controls, as it feels more immediate than mouse and keyboard (and I have been a first-person shooter player forever!)
So in order to have the game fully interactive on the iPad, we just need to write the bit that translates iPad touch controls to feed them into the rest of the gameplay code, which should not take long.
Then, after that, we have to work out exactly how we are going to crunch all the graphical assets down to portable versions and do the streaming in an appropriate way... which is going to be a bit of work. Salvador built a streaming asset system a while back, and we have been using that daily on the PC, but we haven't tried to set it up on a lighter-weight portable system like the iPad yet. We'll see how it goes!
The iPad is one of the major platforms The Witness will appear on. Andy has been working on the iPad port and it is coming along. You can't actually play the game yet, but we probably are close to that. The gameplay doesn't know how to respond to a touch interface yet, though. It is important to me that this interface is very good, so I wanted to start designing for it now, to give me time to wrestle with difficult issues. Since the game runs already on Windows, it seemed natural to start designing the touch interface on Windows.
I am not sure that many people are going to use Windows for touch devices, at least in the Windows 8 timeframe. But at least some people will, so it'd be nice to have a high-quality touch interface built into the game on Windows.
With all this in mind, I started working on some Windows 8 touch hardware a couple of days ago. There is a message format that you can use in Windows 8 to get events from the touch hardware, WM_POINTER, but because I want the touch interface to work in Windows 7 as well, I wanted to use WM_TOUCH messages, which were introduced in Windows 7 and are supported in both operating systems.
It look me less than 20 minutes to get WM_POINTER working. It took many, many hours to get WM_TOUCH working, because WM_TOUCH is some kind of fully insane mess. None of the insanity seems to be documented, or discussed on the internet, so I am laying it out here in case anyone in the future stumbles into this same trap.
A WM_TOUCH message will never be returned by PeekMessage. Spy++ will tell you that WM_TOUCH is being sent to your application, but before the message is delivered, it gets stealthily converted into a mouse motion event. The type of this event will vary depending on whether you have raw input selected: if raw input, it will be WM_INPUT (0x00ff); if not, it will be WM_MOUSEMOTION (0x0200).
If your application handles these events, you may think it's reasonable to complete the handling without calling DefWindowProc. If you do this, you will never see a WM_TOUCH, because something buried down in DefWindowProc stealthily dispatches a WM_TOUCH to your application's windows when it sees a specially-marked WM_INPUT or WM_MOUSEMOVE.
Note that actually, you are supposed to always call DefWindowProc on WM_INPUT (for "cleanup" -- I don't know what this means and why they can't just clean it up when you get the next message -- but whatever); but WM_MOUSEMOVE has no such requirement, and if you believe you handled the event, it is natural *not* to call DefWindowProc. This will result in your never seeing WM_TOUCH. Strictly speaking our game had a bug, in that we were not calling DefWindowProc after WM_INPUT, but the bug was hard to find because it persisted when I tried turning off raw input, because we also were not calling DefWindowProc after WM_MOUSEMOVE (which as I mentioned is, in theory, totally fine.)
Important to note: WM_TOUCH gets dispatched to your windows without you dispatching it! You may think that you can log all messages after they come out of PeekMessage, and that you are in control of your entire message stream, but this is apparently not true. WM_TOUCH will just teleport itself into your window.
In addition to the mouse motion events, your application will get a bunch of other mouse events that are generated by the touch interface, to emulate mouse stuff for programs that don't understand touch. There is no way to tell Windows you don't want these events (unless you do Windows 8 stuff and respond to WM_POINTER; see below). According to Microsoft, the official way you are supposed to detect and ignore these messages is like this:
#define MOUSEEVENTF_FROMTOUCH 0xFF515700
if ((GetMessageExtraInfo() & MOUSEEVENTF_FROMTOUCH)
== MOUSEEVENTF_FROMTOUCH) {
// Skip message
}
I wish I were joking. It's been this way for years; Casey wrote about this in 2008.
This mask works for events like WM_LBUTTONDOWN, but it does not work for the mouse motion events e.g. WM_INPUT. If you want to take input both from touch and the mouse, you need to disambiguate the real WM_INPUTs from the fake ones. Through a process of experimentation with message flags, I landed on this:
if ((GetMessageExtraInfo() & 0x82) == 0x82) { /* ignore event */ }
It seems to work for now, but who knows if that will mysteriously break in some future version of windows.
Nicely, if your application handles WM_POINTER messages, you will not get any of this mouse emulation stuff or WM_TOUCH, so all this ugliness disappears. If it is okay to make Windows 8 your minimum specification, I recommend using WM_POINTER and staying far away from WM_TOUCH.