… which doesn’t look too much different from last time’s. Partially this is because we’ve been working on things that you don’t see from this angle.

Here’s a shot from a different angle:

We still have so much to do!

On my end, I’ve been working for the past couple of weeks on UI-related things. For a week I was doing better touch controls (perhaps we’ll post a video of this soon), and for a week I was somewhat-rewriting the code that implements tracing lines on panels. This can get surprisingly complicated, and I’ve rewritten it a few times. I think this is close to the last time, though.

The game tells me the current puzzle count is 574, which is higher than I expected, but hey.

Feel free to use these as desktop images or whatever. Unfortunately we didn’t mock these up at high resolution; what you see is all there is!

Most of what we’ve been working on lately is not visible in this shot, though! The biggest obvious changes are probably in the forest and beach on the left-hand side.

There are some basic hacks going on; for example, right now, we are not drawing the real sky; it’s just solid blue. Also, right now we are just loading this starting area, but it should not be long before we can load the whole game (Salvador wrote the streaming system a while back and we have been testing it extensively on the PC, so for the iPad it should Just Work. Famous last words.) Also, as you can see, the panel-tracing controls could use some work.

When we can wander around the island generally, I’ll post another video update!

Well, you got a pretty good look at the island in the trailer we recently released in conjunction with the PlayStation 4 announcement. That version has a lot of new modeling compared to the January snapshot, because we were working on the trailer pretty hard. But some new modeling has happened since then, and there are some subtle but nice things to see here.

Since someone is going to ask: The current puzzle count is 499.

Our deal with Sony is a limited-time exclusivity that applies to competing console platforms. Basically it is that you will see the game on the PlayStation 4 for a while before you will see it on the Wii U or the rumored next Microsoft console. (We haven’t been disclosed on Microsoft’s next console, so I don’t know anything about that platform besides the rumors that are on the internet).

This exclusivity does not prevent us from being on the PC or iOS at launch, for example.

People have been speculating that we got paid for PS4 exclusivity, but that is not the case. There was no money involved.

The real situation is that, because we are a small developer, we only have the ability to launch on a small number of platforms at once. We liked the idea of being on a console, and originally we thought we might be on the PS3 or Xbox 360, but eventually we decided not to target either of those, due to the relatively low system specs and the work required to do the ports. After some more time went by, and our release date drifted further into the future, we realized that the next-generation console launch time might be a good time to release the game. (For a while we were hoping to be out substantially earlier than the next consoles, e.g. right now, so we didn’t start thinking seriously about this until recently).

To launch a high-spec version of the game on a console, it came down to a choice between the next Microsoft console or the next Sony console. There were people at Sony who really liked the game and were keeping in touch with us about it, and so we naturally started going to their PS4 developer events, got a dev kit, and started playing with it. I don’t have good communication with anyone at Microsoft right now, but all our technical people like the PS4 specs a lot more than the leaked Durango specs, and we like the positioning of the PS4 (it’s about games) more than what we perceive Microsoft’s positioning is going to be.

So we just found ourselves doing a PS4 port. And once we are doing that, it is not practical for us also to think about another console port. We were going to be a de facto console exclusive for Sony no matter what. As I mentioned, there are people at Sony who are very interested in The Witness, so they somehow percolated up through the ranks the idea of showing the game in the launch show.

Of course, Sony wants that show to point out things that are going to be exclusive or special to their console. If we are in their show, but next week we turn around and announce “we changed our mind, we are exclusive on Durango forever,” that would be a bit weird. So to be in the show we signed a timed exclusivity for competing consoles. But this was just a formalization of something that was already going to be the case. We like the PS4 and we like the people at Sony we are working with, so it was an easy choice to make the agreement.

Exact timing of our release on launch platforms has yet to be figured out. It just depends on how much we can do and when; we’ll know more as we get much closer to the game being done.

(Watch it in HD + fullscreen if you have the bandwidth!)

I didn’t think Witness was going to be an easy project by any stretch, but I certainly didn’t expect it to be one of our biggest creative challenges. I’ve worked on nearly 80 projects spanning feature film, games, ads and other sorts, and my job as sound designer had always been to broaden the experience in every possible way. The Witness inverts this notion.

A tree of any significance in a film or game will have a bird perched on its branches chirping away. Cicadas will buzz. Wind will rustle its leaves. Branches will creak, stress and morph right before your ears. All this happening without you actually seeing it…or really expecting to, interestingly.

In The Witness every element has purpose. A key game mechanic, in fact, relies on the player’s perception as they explore. So, we’re not just talking aesthetics…noise (literally stated) ruins the game’s playability. Noticing, as a buddhist would call it, is key to discovering solutions and even finding the puzzles themselves. Nothing in the Witness is superfluous.

All this said, the island is a tangible living breathing organic place following roughly the same laws of our universe. The trees exist. They have mass and energy. My eyes receive bands of light reflecting off them, and in fact they are quite beautiful. So then what do my ears hear? Silence is not only a bad artistic choice, it’s untrue.

At this point, you’ve probably asked, “What about music?” It’s no spoiler that there isn’t a gigantic speaker system in the sky fed by a mic’d orchestra. So….

None the less, the island is a fascinating place, with lots to see, hear and do. Geoff and I ‘the audio team’ have had an interesting time exploring 1000′s of layers of sonic subtly, and finding ways to engage players to listen ever deeper. This is a unique project for game sound, and we though it would be of interest to share our challenges and discoveries since starting with Jon in 2010 (and of course the challenges yet to meet).

Here’s the interview.

Click here to view the video on YouTube.

The game’s web site.
The Steam page.

The game is primarily the work of one designer, Alexander Bruce, who has been working on it for six years. Players seem to be really into the game, and it’s been the top-selling game on Steam for two days so far.

This hiatus was the primary motivation behind writing my previous entry as a cliffhanger: although I had fixed the bug, I didn’t have time to adequately document the bug via debugger inspection, so it would have been hard to write anything meaningful showing how the bug manifested itself in practice. I wanted to wait until I had a chance to go through every last little bit of it, so I could be as specific as possible. In that sense, it was meant to be a bit of a cliffhanger for me as well.

But as it turned out, it was much more of a cliffhanger than I’d originally anticipated.

Good News, Bad News

The good news: congratulations to those of you who posted “floating point precision” as a suggestion for the cause of the bug. You aren’t any dumber than I am.

The bad news: we’re all dumber than we should be.

For those of you who don’t already know all about floating point precision problems, I’ll explain the problem we all thought we were solving. The original code, verbatim, looked like this:

bool ray_vs_plane(Vector3 *xpoint, Vector3 const &p0, Vector3 const &dir,
                  Plane3 const &plane) {
    Vector3 n = plane.get_normal();

    float denom = dot_product(n, dir);
    float epsilon = 0.0001f;
    if (fabs(denom) < epsilon) return false;

    float t = - ((plane.d + dot_product(n, p0)) / denom);
    if (t < 0) return false;

    *xpoint = p0 + dir * t;
    return true;
}

To fix the bug, I wrote a modified version to use for the transform manipulator:

bool ray_vs_plane(Vector3 *xpoint, Vector3 const &p0, Vector3 const &dir,
                  Vector3 const &plane_point, Vector3 const &plane_n)
{
    Vector3 local_p0 = p0 - plane_point;

    float denom = dot_product(plane_n, dir);
    float epsilon = 0.0001f;
    if (fabs(denom) < epsilon) return false;

    float t = - (dot_product(plane_n, local_p0) / denom);

    *xpoint = p0 + dir * t;
    return true;
}

I’ve done exactly the same operation, but instead of using the d value of the plane equation to solve for t, I just move the ray itself to be centered around a point on the plane, so that the d value is implicitly zero. My reasoning was very simple: if I subtract two things that I know are close together (the camera location and the picked point on the manipulator), I will have less cancellation error than if I subtract two things that may be wildly different in scale (the d value and the dot product of the camera location and the plane normal).

This is pretty weak sauce as far as numerical precision improvements go. I am far from an expert on the subject (read some Forman Acton, for example, if you want to see how the pros roll). But, when I did this, it fixed the bug perfectly, so I assumed, at least for the moment, that it must be a reasonable fix.

But, honestly, I didn’t really have a good feeling about it.

Christmas Stew

Sometimes, when there’s a lot to do, you’ll live with a bug fix because it works, rather than because it makes sense. But invariably, at least for me, I end up going back to look more closely at something if I don’t feel like I fully understood it. I am sure this is mostly just an obsessive personality trait, but regardless, I rarely find the time to be wasted, because I almost always learn something from the process.

In this specific case, I didn’t have time to look into the problem in depth before I left. I planned to do so for the eventual writeup, but of course, I couldn’t do much without being able to run the code, and while I was on Christmas break, that wasn’t an option. Unfortunately, this meant that I had the entire Christmas break to stew about the bug that I wasn’t sure I’d really fixed properly.

From the beginning, the thing that didn’t sit right with me about the fix was that when Jon originally reported the problem to me, he didn’t mention anything about it happening more on the outskirts of the island than on the interior. Jon’s a very experienced programmer, so if something like that were happening, he would probably notice and include that in the report, because he’d know it would help me find the bug faster.

Instead, it seemed pretty clear that the bug was related more to the camera angle than the position in the world. While obviously the camera angle affects the input values, and could have a number of effects on the numerical precision of the program, it’s not intuitively the kind of thing that I would expect. If these sorts of algorithms were that sensitive in this way, I would have expected to encounter similar problems with camera-relative plane intersections many times before — but I never had.

The more I thought about that, the more it gnawed at me that I couldn’t go see exactly what was happening and prove to myself why the fix worked. I did have the game on my laptop, because I’d done some initial Linux porting work, so I realized that maybe, if I was lucky, I could reconstruct the problem without actually being able to load and run the editor.

That is when I became very thankful that I had been writing these blog entries, because it meant I had stopped to take this screenshot before I left, specifically so I could do the writeup over break:

Do you see what I see? That lovely entity number up in the corner is pure gold for step one of reconstructing the problem. Because I could see the entity number, I was able to go into the repository and get the data file that says where that entity is placed in the world. That let me know the numbers of the center point of the manipulator: -304.114, 193.4923, 4.7582.

If you’re an experienced 3D programmer, then you probably are thinking the same thing that I was thinking when I saw those: how the hell could I be having numerical precision problems with such low numbers? The Witness’s scale is one unit per meter, which means that, at the scale of the dragging we’re talking about, there should be absolutely no problems with solving a single plane equation, regardless of how sloppily you did it.

But if that was the case, then how did I fix the bug? This lead me to step two of reproducing the problem: writing a test program. I wrote a little C program that would take numbers around that point and, for a large set of programmatically generated planes and rays, perform the intersection using both the old ray intersection routine and my “improved” one.

Perhaps unsurprisingly, these results reinforced my suspicions that there wasn’t enough of a difference between the two routines to account for the original bug. In all the tests, my “improved” routine was no more than one decimal digit more precise; so if the original routine produced intersection points that were actually, say, 0.0001 units away from the plane, the new routine might produce ones that were 0.00001 away.

Clearly, the fix just plain shouldn’t have worked. But it did.

Why?

The Wrong Part of the Post

I would love to be able to say that I guessed the real reason before stepping through the code, but I absolutely didn’t.

When I returned from Christmas break, I eagerly booted up the Witness and stepped through the code very carefully, keeping accurate track of all the numbers as I went, waiting to find that super-elusive numerical thing that I had overlooked. To my complete surprise, it turned out to have absolutely nothing to do with precision.

In fact, somehow, although I didn’t know it at the time, I actually did write about the root cause of the problem in my original post. It just wasn’t in the part that I thought. Indeed, rather than Always Look Behind You foretelling the eventual solution, it was Foreign Languages that contained the hint: due to my naivete with The Witness codebase, I misunderstood what it was doing with planes. It turns out that this one line from the original routine was the entire culprit:

    Vector3 n = plane.get_normal();

You will notice that it only occurs in the buggy version of the ray intersection code. In my version, I couldn’t use the plane structure, because I didn’t want an “a, b, c, d” representation of the plane. I wanted “a, b, c, p”, if you will, where p is an actual 3D point on the plane. So I never called get_normal().

Why does this matter? Well, upon stepping through all the code, I realized that get_normal(), being true to its name, takes the unusual step of normalizing the a, b, c values of the plane before returning them to you as a vector. This means that the a, b, c values of the plane that you pass in when you create the plane aren’t the ones you get out.

That wouldn’t be a problem if it weren’t for the fact that, when you create one of these planes, it does not normalize the values you pass in:

    Vector3 plane_n = cross_product(camera_back, translation_axis);
    plane_n = cross_product(plane_n, translation_axis);
    grabbing_plane = plane_from_point_and_normal(hit_pos, plane_n);

This is my fault on its face, because hey, it does say normal, and normals are traditionally supposed to be normalized. But in keeping with my reference to foreign languages, this is something I really just didn’t think about, because I am used to the way I’ve always done things with planes: nothing is normalized automatically, everything is just left unnormalized, because back in the day, normalization was very expensive and you never did it in a math routine unless it was absolutely necessary.

So plane_from_point_and_normal was actually doing what I thought it would: it just stored plane_n as the a, b, c values, and computed d with the inner product. But get_normal() was not doing what I thought it would. Rather than just returning me the vector {a,b,c}, it was normalizing them before giving them to me, which meant all my calculations with them would no longer agree with the d value computed in plane_from_point_and_normal.

And therein, clearly, lies the entirety of the bug.

Just to underscore how much of a native language issue this really is, when I showed Jon the preview of this article, he said, “If you are thinking of the plane as a geometric object, of course the normal is unit length, what else would it be?? But if you are thinking of it as some specific equation all the time, then it is natural to care about the a, b, c, you put in.” So I can only assume, were he using my math library, he would’ve been tripped up by the fact that I don’t normalize normals automatically, exactly the inverse of the problem I encountered with his.

I honestly don’t believe that this sort of thing is a case of us being presumptuous or sloppy; it’s just the price you have to pay for being able to program complex things at a reasonable speed. If we constantly have to think about what it means to work with a vector, or a plane, or a quaternion, we’re going to be a lot slower. It’s much more efficient to have a consistent perspective, and always rely on it.

And of course, the cost is bugs like this — but that’s a very small cost relative to the benefits.

Partial Credit

So, where does that leave you?

Clearly, automatic partial credit goes to everyone who wrote in suggesting that numerical precision was the problem. You came to the same conclusion I did with the information I knew at the time, which was all you had to go on. You certainly can’t be blamed for not finding the real bug since you would have needed the source code to the Witness math library to even see it. And it is true that the numerical precision of the routine could have been improved, as I believe I was able to verify separately with my test program.

But by the same token, nobody gets automatic full credit. There were a lot of reasons why, even with only the information in the original article, you could have said, “I don’t think it’s numerical precision, Muratori.” Maybe I am ascribing overly heroic attributes to the man, but I feel like Forman Acton would have known, from looking at the plane intersection snippet, that there wasn’t any way to lose enough precision in there to account for the bug as I described it, at least not in any way that would have been substantially improved by the suggested fixes people wrote in (or the fix I actually did, for that matter).

So in order to actually get full credit, you’re going to have to earn it, I think, but on the honor system. Ask yourself, if you had encountered this bug, and you had “fixed” it as I had, would you have gone back and figured out what was really happening before moving on to something else?

If you can honestly answer yes, then you get full credit, for this is about as good as you can hope to do as a working programmer, I think. You’re no Forman Acton, of course. . . but then again, who among us is?

Up until now we have not been in that kind of work mode, because we were figuring out the game, and in that kind of mode, I find schedules to be anti-quality. But once you have everything figured out and need to produce a volume of work, schedules are pro-quality, since they help you gauge how well you are doing, whether you need a reality check on the rate or quality of things you can produce, etc.

Luis has made this handy map that we are using to partition up modeling tasks, which I have then edited to remove spoilers:

I like looking at this map because it reminds me how many distinct areas there are and how they relate to each other.

Today I was showing the game to someone who had not seen it before, and that’s always interesting because it helps me see the game through fresh eyes a little. It really feels unlike any other game in the way it is an Open World but also a Small World, where you can be in one location and see several neighboring locations, and that helps keep you oriented, but it also helps you reflect on what you have done and orient on where you are in the course of things you are planning to do.

We still have a ton of work to do, though, and I was reminded of this as well. Not just in detailing all these areas, but in filling out the story, crafting details that further help the areas relate to each other, etc.

It is going to be an interesting time!

Jeff asked me about this part of the iOS App Store’s Developer Terms of Service:

We view Apps different than books or songs, which we do not curate. If you want to criticize a religion, write a book. If you want to describe sex, write a book or a song, or create a medical app. It can get complicated, but we have decided to not allow certain kinds of content in the App Store.

And here is what I said:

I think this is the wrong attitude about games, but look, ultimately it is game developers’ fault, not Apple’s. Apple is treating games as shallow commercial entertainment experiences because they have been taught by game developers that that is what games are.

If we had built a world where games routinely work with serious issues in ways that people care about, Apple would not be able to take this stance, because it would not make any sense.

Why do they say “If you want to criticize a religion, write a book”? It’s because it’s obvious that banning books is bad, because there have been a lot of books that people find important (and we have had many cultural cycles involving people attempting to ban books, and culture has worked out ultimately that this is not a good thing). Games do not have this history. Right now Apple thinks a game is Angry Birds or maybe Infinity Blade. So who can blame them?

Apple may be badgered enough to change their policy someday, or they may not. But that doesn’t matter very much because really it is just a reflection of the general cultural idea about what games are. The only way that idea will change is if a lot of developers make a lot of serious/deep/honest/touching/intrepid games for a long time. I don’t know if that will ever happen. How many games can you think of to which you can seriously apply these adjectives? Certainly to none of the top-selling games on the App Store, and certainly certainly to none of those big free-to-play games that are raking in all the cash.

So, game developers are just sort of reaping what they have sown. What else would you expect?

I have not played Endgame: Syria but maybe it is a step in this direction. But to change these attitudes we need a lot of steps, consistently, not just a token step now and again.

Current puzzle count: 505.

The Background

Way back in 2008 when I came upon the core idea of this game, the design was the exciting part. Though I started in games mainly as a technical person who was excited about technical challenges, I have shifted into a mode of thought where, when I program, it’s mainly just to get the design done in the most straightforward way.

Since Braid I have viewed programming as mostly pragmatic: I know what I want to do, and I know what kinds of things I have to type in to make it happen, but for the most part, the interesting mental challenges are in design, and programming is rote execution. Once you have enough programming experience, I think 95% of writing a big program is like this. There have been a few times during The Witness when I spent significant effort solving technical problems where I had no idea what the answer would be in advance, like with the terrain manipulation, but even in these cases I treated the technical development in a very pragmatic way: the goal was to get something that is good enough for current needs, not to design a shining jewel of an amazing technical system (back when I was a younger game programmer, I was trying to make amazing technical systems all the time, and it came at the cost of actually getting games done).

Despite the fact that I am trying to focus on design, in order to make this game a reality, I had to start a company, bring people onto the project to build the game, etc. This means that nowadays I am not just wearing the programmer and designer and business development hats, I also wear the producer hat and the (admittedly small at present) HR hat. Maybe if The Witness does well, we can expand the team a little bit so that I don’t have to do these things, but for now, that’s the reality.

We are very laid back and unstructured in the way we do things, but even so, I feel there’s a heavy subconscious load involving running the company, making sure nobody’s work is blocked, dealing with money all the time, etc. I often find it hard to do deep design thinking when I am in the office, if only because all these issues are just ambiently around. So, in terms of design, my most productive days tend to be work-from-home days (which for me includes most weekends).

All this up til now is just to explain that if I take a vacation seriously, it can be very relaxing, because that subliminal load is gone, or at least mitigated for a while. The activity of the rest of the company is mostly paused, which removes urgency from my own activities.

At first I didn’t know how I would spend vacation; at first I thought maybe I would actually treat it the way most people treat vacation, and not work. Then I thought, hmm, there is a certain sector of the game that I might like to work on, that is definitely very important, and I should get a bunch of that done while everyone is gone, so that when they come back it will be figured out. (I don’t want to go into detail about what that section of the game is, because I am avoiding spoilers).

The Vacation

But, inspiration is a funny thing. I worked a little bit on the issues I had planned to work on, but for the first bit of the vacation, up through the 25th, I found myself most motivated by cleaning up old nagging issues on my to-do list, and shipping this improved version of the game to our very few beta players. This involved stuff like improving the feel of tracing lines on the puzzle panels.

A peek at the export script, used to package the game for end-users.

After shipping that beta, I found my attention most strongly drawn toward an unexpected part of the game: an optional area, one that we could completely cut from the game and nobody would know, yet at the same time, it’s an area that adds to the overall conceptual arc of the game in a strong way, and would be very rewarding for the small percentage of players who get deeply into the game. (Again, I apologize for the vagueness, but let’s avoid spoilers).

I had roughed out a version of this area a long time ago — maybe a year or two ago — and had always found the basic idea very exciting. But back then the area wasn’t done enough to prove the concept, and over time it decayed while the rest of the game was being worked on (which is natural on a big project like this). But, perhaps because it’s an area of the game that nobody else on the team really knows about or understands yet, I had a great motivation to spend vacation time on making this area good. Now that I think about this again it makes a strange kind of sense: when development is pretty much paused, it is somehow appropriate to work on something that nobody else really knows about.

At first I thought the task would just be to make the old stuff work again and then do a straightforward job of the extra functionality I needed to complete the area. Thinking about the programming I would have to do to complete the area, I saw it as a necessary bummer (“I want to be done with this game, but ugh, there is more programming I have to do that isn’t even on the mainline to all the things that really need to be done.”)

But I started programming that extra functionality, and 10 minutes into it, realized I was not so excited about the design part — it seemed like a very rote execution, like anyone could do in any game, and The Witness ought to have something better than that. Because there was no pressure, and because I was free to do whatever I wanted, my mind was open enough to instantly see an alternative that would be much truer to the essence of the game. The chance to do this better thing was very exciting. (We’ll come back to this subject of being able to see ideas, later).

So I started programming this more-exciting version of the functionality. Then I saw how I could make it even more interesting, and I would start doing that. Then I would see how to make it even more interesting, though that would introduce a lot more programming work. If I were feeling the regular pressure of the game schedule, at this time years in when it is time to be finishing the game, I might have said, “naah, we don’t have time add that extra work”. But because it was vacation the work was free, so I said screw it.

I worked in a fiercely productive way for a couple of days. New ideas would come. They would raise new issues. I would program some things, and in that way that programming often goes, new problems would come to light that I hadn’t thought of, and those problems introduced even smaller 2nd-order problems when confronted. But by this time I was fully energized and just had dogged determination and was fully enjoying the whole process. In the end I programmed at least 5 times as much functionality as I had planned, but rather than being bummed about it, I enjoyed the whole excursion.

Those who know me will tell you that I am not predisposed toward painting things in an unduly positive light — but I will tell you that these past four days or so have been amazing and wonderful. Here’s a shot from what I did (the visuals are ‘programmer art’, as usual):

It is hard to tell you why this is so cool without huge spoilers.

It’s so amazing and wonderful because

it’s been a long time since I worked this hard and this effectively. Keeping in mind that memory is sometimes not as reliable as we think, I would tell you that the last time I worked this hard and this well was when starting my first, ultimately doomed, company (with a friend from college and a 3D artist we recruited from that same school and unfortunately put through years of meager-wage toil).

Here are some shots from the first professional game I ever worked on, back before we had a concept of ‘indie development’. Software rendered, subpixel-accurate texture mapping, assembly-language inner loop that commandeers ebp in order to utilize a precious extra register, a la Mike Abrash, and oh yeah, it was multiplayer-only, 32-player, over 9600 baud modems:

We were working on this game from 1996 to 1999 or so. 1996 was the year I got into professional game development, and in hindsight, it appears to have been the hardest possible time in the history of video games. Indies of today, y’all have no idea how much easier it is to make games now than it was in 1996. (I hate to be an old fogey who’s all “back in my day, we had to walk to school in the snow, barefoot, uphill both ways,” but in this case it is absolutely true.)

That game was never very successful. It had design problems, but also, we didn’t really know how to talk about or promote a game; lastly, this was before many people played games over the internet, so the potential audience was pretty small. (Trivia Tidbit: There was another online game being supported by the same publishing service as our game, and this other game was more popular, despite being relatively simple in a technical sense, which annoyed the hell out of us on a subliminal level that we wouldn’t totally admit. The guys making this game were John Vechey and Brian Fiete; our producer at the publisher was Jason Kapalka, who later went off with these guys to start a company called PopCap, which last I heard had done okay.)

Back in those days I worked very hard. When we started the company I knew nothing about building 3D games; we learned very quickly. I would long hours, late into the night, intensively. At some point we all burned ourselves out, I think.

After we closed down that company, I went on to do a number of contractor things, game industry lecture things, magazine article things, and working on various independent projects that I never finished. Through that time, I was still very burned out. If you have never experienced burnout, it’s hard to explain. Burnout is not just “I don’t feel like working right now”; it is about your mind refusing to permit you to work, because it has seen what happens when it lets you work.

Eventually, I swore that I would design a very good game and finish it and release it to people, and the result is a game that was successful and that many people like:

A main part of the reason this game exists was that I instilled a certain discipline in myself: I wasn’t going to do this game because I was excited about a technical thing, or excited about programming; I was going to minimize the amount of programming, firstly so that the game would get finished at all (I had plenty of experience by this point at not-finishing-projects, as did many of my friends; it was an endemic problem), but also so that I could focus appropriately on the design and build something very special.

Despite the fact that I kept the game technically simple, there was still a huge amount of programming to do. But thankfully, the strategy worked and the game was completed. Eventually.

Even though I’d made a successful game, I was still burned out to some degree. In 1996-1997 I was working at full capacity and was very productive, at least inasmuch as one might expect for a programmer of that age who was first starting to get real-world large-project experience. But after being burned out at that time, I never again worked at full capacity.

What I am getting at

I am saying all these things just as a way to get around to explaining that, during this Christmas vacation, I finally recovered fully from burnout. For about 3 days, I was working at 100% capacity, which I have not done in 15 years. It felt wonderful to have that back, to not succumb to the story that maybe game design and programming are like math of physics where you peak at 25. These few days were way better than when I was 25, because now I am substantially better at programming, and I am good at design also, and games are so much easier to make than they used to be; so the design cylinders are firing and then they turn it over to the programming cylinders, etc, and very quickly a lot of high-quality work gets banged out. And it’s not just arbitrary good work, but work that helps to round out the game that I already feel is the best thing I have ever worked on, that helps put grace notes in just the right places, in the manner of exploratory game design that Marc ten Bosch and I spoke about at Indiecade 2011. So this period of intensive work was seasoned and tempered in a way that was not possible when I was 25.

It has been a wonderful time and I look forward to doing more of it. Part of what allowed it to happen this way was that lack of pressure that I have mentioned, due to being on vacation. When a design idea would come to mind, and I would see that it might be cool but would lead to a lot of work, well, this week I was able to let it happen, and I was able to enjoy the challenge of going off on a minor programming rathole in order to do something really good in the design. This is to some degree a reversal of the discipline I instilled to get Braid done; you can think of it as a correction term to the equation. The 0th order term is knowing how to program and design; the 1st order term is having discipline to be able to execute efficiently without being carried away with ideas that will scuttle production; but the 2nd order term is that it is joyful and interesting to have some wiggle room in there, sometimes eroding away a bit of the 1st order term.

There is a kind of open curiosity about programming that I haven’t had in a long time, that I allowed myself to re-discover this week in the freedom of exploring that 2nd-order term.

(Richard Feynman has an anecdote about how he was burned out on physics, and how he started to get over it, which I think bears some similarity).

I am not explaining any of this very well. But I am doing the best work of my life, right now, and it feels great, and for the first time I could see myself working in this same fashion for the rest of my life and feeling happy that I am operating at something like my full potential. So there you have it.

I mentioned earlier that I would talk further about the ability to see ideas, but I guess I was lying, because this is already too long and rambling. I’ll save that for a future post.

It’s worth pointing out that I started Braid while working on what was supposed to be a vacation. And back when I got the impulse to start the Experimental Gameplay Workshop, I was on a plane back from a vacation to Africa. So apparently vacations are good times for me, even if I don’t treat them the way Normal People do.

I do not know who made this rock or how long it has been there, but when I first happened upon it, I found that it resides in a nicely isolated part of the island where there’s nothing else in view, so the frame rate is always high even in full debug mode. Plus, it’s big, unobstructed, and easily pickable in the editor without fear of selecting any other objects. These properties make it perfect for testing editing features in progress, so the rock and I have become fast friends.

Normally, I do not want large rocks to move around. If I’m in the game engine, and a giant rock starts moving, I know I’m in for some debugging work. But if I’m in the editor, and I tell a rock to move, I expect it to move. That’s the whole point of having an editor. So when this cliffside rock, which otherwise had been so pleasant, decided that it would only move in certain directions, I knew I had a bit of a mystery on my hands, one that eventually forced my brain to remember some very important things it had forgotten in the ten years since I last did any serious 3D graphics programming.

Foreign Languages

Working in someone else’s codebase is much like moving to a foreign country that happens to use the same alphabet as you do, but not the same language: technically, you can read, but you don’t really have any idea what most things mean. At first, every little thing you do is a painstaking linguistic procedure, and you have to constantly check and recheck things to make sure you actually know what’s going on. Over time, as you gain more experience, you start to intuit the way things work, and you’re able to operate at closer to your full speed.

I’m used to the process of working in unfamiliar codebases since I’ve worked with so many of them over the years. I tend to approach each new one the same way: first, I write something very isolated, and keep it walled off. This lets me start making some contributions right away, while I gradually learn how to interface with the rest of the code. In the case of The Witness, this was Walk Monster. Next, I try to do some performance optimization or bug fixing, so I learn to work locally, but directly, with existing code. That was fixing the five-second stall. Finally, I start picking real features to add to other people’s code, preferably in the systems that I will most likely need to learn well.

Since I’m working on the collision system, and at this point I am certain there will need to be some interactive tools for it, I picked the Witness editor as the best place to start adding real features. After playing with it for a while, I selected two features I thought would be good additions from both the standpoint of learning the code, and for be generally useful for people working on the game. The first feature was compatible camera controls, and the second was a translation manipulator.

By “compatible camera controls”, I mean camera controls that work like common 3D art packages. My understanding was that the Witness team did their modeling in Maya, so I put a toggle in the editor that, when enabled, allowed you to use ALT-drag camera movements that replicate the way Maya moves the camera. After checking this in, I learned from the artists that some of them prefer 3D Studio MAX, so I added another button that emulates its camera controls (once you do one, it’s very little incremental cost to add others, as they’re really all just other sets of bindings for the same movements). All of this went rather smoothly.

Unfortunately the second feature, the translation manipulator, did not go so smoothly at all.

The Translation Manipulator

By “translation manipulator”, I mean the standard three-axis overlay that modern 3D packages have where you can click and drag on any axis and move your selection along that axis. Depending on the package, there are also little squares on each plane that allow planar movement. I chose to include these, but to allow them to be toggled separately since not all packages have them and I didn’t want them to be in the way if an artist wasn’t accustomed to using them.

Compared to the camera controls, it was moderately more work to understand all the intricacies of the editor code involved with translation. But I worked my way through it, and eventually I had a translation manipulator working reasonably well in my test world, which was a completely empty world with just a few boxy entities that I could move around to see if everything was working.

When I was confident everything was solid — undo was working, occlusion was working, etc. — I tried it out on the full Witness island for a bit. Nothing seemed immediately wrong, so I checked in the feature on a toggle switch that defaulted to “off”, and asked Jon to check it out when he got a chance. I figured it was better not to check it in enabled by default and have the artists all start using it, only to find out it had some critical bug that would corrupt things in some irreparable way, losing lots of valuable work in the process. Since Jon does a lot of work in the editor, and he wrote a lot of the editor code originally, I figured he would know the things to try that might break so we could fix them before any real damage was done.

When I heard back from Jon, he said it did seem to be working correctly, but sometimes “moving along the Z axis wasn’t working”.

The Occasionally Movable Rock

I was not at all surprised to hear that Z-axis translation was unreliable. This was something I was expecting to have happen. Surely something was broken in the way my code interacted with the existing planar movement constraints, which were buttons the editor had always provided and which were the only way of constraining interactive movement before I added the translation manipulator. But I checked the code thoroughly, and I just didn’t see any way that I could be mishandling the Z axis in the intermittent way Jon had reported.

I tried playing around with movement more in the editor, to see if I could reproduce the problem myself. Soon, I started to notice that when I played with the solitary rock in the ocean, it wasn’t really “sticking” to the cursor as well as it should be; the code I had written was supposed to feel as if you had “grabbed” the manipulator, and it should follow your cursor rather exactly. It wasn’t doing that.

Then I noticed something even stranger. Depending on how I rotated the camera, I was able to find angles at which Y-axis dragging moved the rock extremely slowly even for large cursor movements. At one particular camera angle, the rock refused to move along the Y axis at all! For example, at this angle, I could move the rock along the Y (green) axis, albeit slowly:

and at this angle, I could not move the rock along the Y axis at all:

even though the other two axes worked fine, and from other camera angles the Y axis worked fine, too. Since the manipulator supported entity-relative axial movement as well as world-relative axial movement, I tried switching to entity-relative axes to test if there was something wrong with the Y axis specifically. I rotated the rock such that the Z (blue) axis was pointing in the direction that the Y axis pointed previously:

To my surprise, it now refused to move along Z! So it really had nothing to do with which semantic axis you were moving, it seemed to be the direction in the world and the angle of the camera alone that mattered for reproducing the bug.

Looking for more ideas about what the bug could be, I tried loading the empty test world again and playing around with the manipulator there. Again, to my surprise, it was completely impossible to reproduce the bug. No matter what camera angle I used, or where the axes were pointing, the always dragged perfectly, and seemingly at the correct speed to boot.

What on earth was going on? Clearly, it was time to think more deeply about the code and the behavior it was exhibiting.

Ray Versus Plane

In most 3D editing operations, you must confront the problem of turning a 2D cursor position into something useful in three dimensions. Since a 2D point on the screen mathematically represents a ray in three dimensions (everything in the world that is “under” that point from the perspective of the user), this typically involves taking the ray and intersecting it with something. For example, to implement picking of objects in 3D, you typically intersect this 3D ray with the objects in the world, and you see which intersection is closest. That will be the closest object under the cursor when the user looks at the screen, and therefore the one that they most often expect to pick when they press the mouse button.

My translation manipulator also works by intersecting the ray implied by the cursor. Knowing which axis the user clicks on is analogous to the picking problem, and I essentially implemented it that way: I pretend that there are three long, skinny boxes that align with the axis arrows, and I test to see if the cursor ray intersects them.

Picking the axis is straightforward, but things get more complicated once dragging begins. Although I know which axis the user started dragging, in order to actually move things properly, I have to have a way of computing exactly how far they have dragged the object along the selected axis. Intuitively you might think well, why not just extend the box used to pick the axis to be infinitely long along the dragging axis, and continue intersecting with it as the user moves the cursor? That’s not a bad idea, and it would work, but it isn’t the best solution.

The reason is because it doesn’t feel very good to use. Although it will feel fine if the user drags right along the axis in question, a lot of times users don’t do that. They tend to drag erratically, and pay more attention to the object’s position than how closely they follow the line implied by the axis. This quickly moves the cursor off the imaginary box entirely, so the ray implied by the cursor won’t intersect it. This causes the manipulator to stop working until the user moves the cursor back onto the axis, resulting in a sticky and unpleasant user interaction.

What we’d rather have happen in this case is for the dragging to happen as if the user had moved the mouse to the point on the axis closest to where they actually dragged, even when they’ve wandered quite far from the axis itself. To see how to solve this problem geometrically, imagine a user dragging the mouse along the X axis of a 2D plane. If the user starts dragging erratically, and moves along Y as well, their cursor will end up out in the middle of the plane somewhere. Figuring out where they would have been on the X axis is quite simple, though: ignore the Y coordinate, and just use the X coordinate. Nothing could be simpler.

But how do we do this operation for an arbitrary 3D axis and a ray extending from the cursor? It turns out you can do it exactly the same way. All you have to do is pick a 3D plane that happens to contain the dragging axis, then intersect the cursor ray with that plane. That gives you a nice 2D coordinate on a 2D plane, where the dragging axis is analogous to the X axis in the simple example. You then “throw out the Y”, and are left with a drag that always feels nice in 3D, sticks the object tightly to the cursor, and does not require the user to drag close to the axis.

You don’t need any fancy math to do this. It’s entirely constructible with simple operations available in any math library: dot product, cross product, and ray-intersects-plane, all three of which were already available in The Witness codebase.

So my translation manipulator was built entirely out of the most basic 3D operations in any math library. Yet somehow it was exhibiting very odd behavior atypical for a simple system. How could this be? What was I missing?

Debug Rendering

My first thought was that I must be picking the dragging plane poorly. Constructing planes in 3D with only partial information is always a little dicey; you need three non-collinear points to define a plane, but often you want to construct one with only one or two points. The translation manipulator is a perfect example: it needs a plane to operate, but it only really has a line to work with, which is equivalent to only two points. There are, in fact, an infinite number of planes that pass through the dragging axis of the translation manipulator. It’s much like a waterwheel: the dragging axis is the center axle, and each blade of the wheel is another valid choice for a plane.

So how do you pick one? The best method I’ve found for translation manipulators is to use the location of the user. Since we want the user to be able to drag around on this plane easily, it makes sense to say that, of all the planes containing the dragging axis, the best one is the one that is most visible to them. This avoids the potentially bad cases you can hit if you pick something independent of the user’s location, such as the resulting plane being edge-on or nearly edge-on so that the user cannot actually drag along it effectively.

Since even simple 3D math like this can be difficult to verify by inspection alone, my first inclination was to add some debugging rendering to verify that I was picking the dragging plane properly. Since the camera angle clearly mattered, and the plane was one of the only things dependent on that angle, it seemed like a likely culprit. So I tried drawing the normal to the plane in yellow at the base of the manipulator:

Even in the cases where the dragging failed, the plane normal looked right. Maybe the positioning of the plane was wrong? I tried computing the location of the plane directly, by moving along the normal by the plane constant d and drawing the normal there. This, of course, put the normal nowhere near the manipulator, so I had to look around for it (the plane equation gives you the point closest to the origin, which has no real relationship to where the manipulator might be, other than that they both lie on the same infinite plane). But once I found it, even though it was far away, looking back toward the manipulator it still did seem like it was the right plane.

Next I moved onto the intersections. I tried drawing the intersection point that I had computed when picking the axis in the first place, and also the point that I used to compute the plane constant in the first place. Both looked correct.

But then I tried drawing the actual intersection computed at each drag update, and that didn’t look correct. In fact, it didn’t really “look” at all. Although I was specifically telling the renderer to draw a white sphere where I’d found the intersection of the cursor line with the plane, no white sphere appeared. I checked the code again, and it looked correct, so I tried doing a few more test drags, this time from angles that had seemed to be working before. Now, the sphere showed up, although it was bigger than I thought it should be, which was odd.

I tried rotating the camera. I wrote the code so as to leave the sphere rendering wherever the last dragging intersection occurred, so if I stopped dragging, it just stayed where it was. This allowed me to rotate the camera and see precisely where the sphere was located, and as I did, I realized that the reason the sphere appeared too large was because it wasn’t even close to being on the plane I thought I was using. It was much closer to the camera. Weirder still was that, at the buggier camera angles, the sphere was jumping around erratically, and wasn’t really sticking to the cursor, which it always should do no matter what plane was being used. I was, after all, intersecting the cursor line with a plane. How could the intersection point possibly lie anywhere but directly under the cursor, regardless of the parameters of the plane?

It was at that moment that my math brain finally woke up. It had been sleeping for about a decade, which is how long it’d been since I’d done any real mathematical programming. In an instant, I immediately knew both what the problem was and how to fix it.

Always Look Behind You

The problem was embarrassingly obvious when you consider the clues I’d been generously given and yet completely ignored. Remember when I said I drew the plane normal at the location indicated by the plane constant, and I had to look “far away” at the manipulator to see if it looked like it was in line with the plane? Well if the generally erratic behavior of the manipulator hadn’t been telling enough, that little episode should have been the dead giveaway.

At this point, every serious 3D programmer who hasn’t been doing other work for ten years surely knows what the bug was. But for the sake of people who are less experienced, I decided to end this article with a bit of a cliffhanger.

I am going to tell you that you can find the bug by looking at this simplified version of the ray_vs_plane function in The Witness that I was calling (the real version does some epsilon checking in case the ray and plane are parallel, but otherwise it is identical):

Vector3 ray_vs_plane(Vector3 ray_origin, Vector3 ray_direction,
                     Vector3 plane_normal, float plane_d)
{
    float denominator = dot_product(plane_normal, ray_direction);
    float numerator = dot_product(plane_normal, ray_origin);
    float t = -((plane_d + numerator) / denominator);
    return ray_origin + t*ray_direction;
}

And I will also show you this screenshot of what it looks like if you turn around 180 degrees from the rock and look back at the island itself:

But, until next time, I’m not going to tell you what the bug actually is. Can you find it, and figure out how to fix it, before then?

Granted, it is not uniformly horrific. It ranges from only slightly scarey (xinput, comdlg32) to full psycho (DirectShow, Event Tracing, TabletPC, etc.). But no matter which part of the spectrum you encounter, programming on Windows tends to be about doing a lot of unnecessary research, experimentation, and debugging due to a confluence of poorly designed APIs.

Since time spent debugging Windows issues isn’t often chronicled, I thought I’d go ahead and describe my experience this Saturday of tracking down a Windows bug in The Witness.

The Symptom

Apparently, not everyone on The Witness team was seeing the symptom. But trust me, it was there.

On both Jon and my machines, there was a period during the initial loading of the game where you couldn’t do things like ALT-TAB back to the debugger. If you set a breakpoint then ran the game, when the breakpoint eventually hit and the debugger came to the front, it would wait about five seconds before it allowed you to actually interact with it. Sometimes, in rare cases, it would go into some sort of psycho mode whereby mouse movements stuttered, and the computer would literally beep at you as the mouse moved, for several seconds. Then everything would return to normal.

While not technically a showstopper, this is the kind of bug that’s worth fixing if you can, because it costs you real development time every day. Five seconds wasted every breakpoint is bad news, and people often underestimate the effects of frustration that can build up over time due to flaky and unpleasant development environments. So I decided to try to track it down.

Because Windows is a closed-source platform, there’s no great way to track down problems of this nature. However, having programmed on Windows for over twenty years now, I have a nose for it, so I felt like I knew the best place to start: since there was definitely a UI blackout somewhere during startup, there had to be something occurring early in the execution of the program that was causing at least that particular symptom. So before doing anything else, I started stepping through the entire startup sequence of the program to see if anything out of the ordinary struck me.

I very quickly came upon this code:

// @@ The hook appears to install properly, but is never invoked :(
keyboard_hook = SetWindowsHookEx(WH_KEYBOARD_LL, LowLevelKeyboardProc,
                                 GetModuleHandle(NULL), 0);

Now, I don’t want to make it sound like the “find the source of the problem” phase of Windows debugging is straightforward. It usually takes a very long time, sometimes multiple days. But as luck would have it, this was literally the first piece of code I suspected, and it turned out to be the culprit.

I commented out the SetWindowsHookEx, ran the program, and poof! All the symptoms were gone.

The Problem

From experience, I’ve learned that it’s always best to fully understand a problem before you fix it. If you just patch over its symptoms but never figure out what the problem really was, it will often come back to haunt you.

In this context, that meant two things: first, I should figure out specifically why the keyboard hook was causing the UI stalls; second, I should determine why the code had been trying to call SetWindowsHookEx in the first place, even though apparently it never quite worked (according to the “is never invoked” comment).

The first part turns out to be obvious in hindsight, but it took me a little while to put all the pieces together and be sure I had a solid explanation for what we were seeing. As with any puzzle, looking at the finished picture is easy, but figuring out what that picture might be when all you have are little pieces coming in one at a time is much more difficult. Here’s the best picture I could come up with, but of course there’s no way to completely verify it external to Microsoft:

Normal Windows hooks are pieces of code which execute in the address space of the eventual message recipient, not the installer of the hook. It is for this reason that, normally, if you want to install a global hook, you must put the hook code in a DLL, because that DLL must be mapped into the address space of every running executable on the machine that can receive Windows messages.

But low-level hooks are completely different. A low-level hook like WH_KEYBOARD_LL is a piece of code that resides in only the process that registers the hook. Windows remembers which thread registered the hook, and when any other executable is about to have a hooked message posted to its message queue, Windows actually waits until it can switch contexts back to the hook thread, run the hook in that context, and then finally deliver the message.

This leads directly to the symptom that we were experiencing. When the game is loading, its main thread isn’t yielding to Windows frequently like it is during normal play. Since the main thread is the one that registered the hook, Windows must wait for that thread to finish all the load-time work it was doing and call a function which yields to Windows in order for Windows to actually call the hook procedure. Since no keyboard processing can occur until the hook procedure has run, the keyboard becomes unresponsive as messages back up waiting to be processed through the hook. This completely explains the UI problems during startup, and fits with the oddity that the mouse never seemed to be affected, just the keyboard.

When the game hit a breakpoint, the keyboard hook couldn’t execute at all because the debugger had halted the thread required to execute it. This would have meant that keyboard input would be completely halted, and the debugger would never be able to receive keystrokes, if it weren’t for the fact that Windows actually uses a timeout on hook calls. If the timeout value is exceeded waiting for the hook thread to become available, the hooking executable is assumed to have crashed, and it will silently remove that hook and continue with normal operation.

That timeout value? Five seconds — right in line with our observed pause (technically, it’s whatever is set in the registry under HKEY_LOCAL_MACHINE\Control Panel\Desktop\LowLevelHooksTimeout, but the default is five seconds).

So, no mystery as to the pathology. But what about the second part? What was the code actually trying to do?

The Windows Logo Keys

Although SetWindowsHookEx might seem like an odd call to make in a game executable, it’s actually quite common. It’s there to prevent the Windows logo keys on modern keyboards from ruining full-screen games. Because the keys are placed in a location that’s easy to hit by mistake, many gamers would accidentally hit them, causing their games to be instantly deactivated by Windows in favor of bringing up the start menu. So widely acknowledged was this problem that, when these keys were first introduced, Microsoft itself published a recommended work-around. From Disabling Shortcut Keys in Games:

This article describes how to temporarily disable keyboard shortcuts in Microsoft Windows to prevent disruption of game play for full screen games… Use a low-level keyboard hook to filter out the Windows key from being processed.

The article goes on to show sample code that is basically the same code that was being used in The Witness. It’s a simple SetWindowsHookEx call with WH_KEYBOARD_LL whose hook routine does nothing but block VK_LWIN and VK_RWIN keys from being processed. The only difference between the code in The Witness and the code in the sample is that The Witness would conscientiously remove the hook when it no longer had focus, and then reinstall the hook when focus was regained. The sample code, by contrast, just set a global variable.

So if this was what the code was doing, why the frowny face? If you look back at the comment, it states very clearly that the hook was never being invoked, even though no error message came back from Windows. And, although I normally remap my Windows keys via the registry, I went ahead and mapped them back to see if The Witness was properly blocking their effects.

Much as the frowny face foretold, it wasn’t.

Obeying the Static Discipline

I don’t know anything about digital circuit design, but I once watched a lecture by Gill Pratt where he explained something called “the static discipline”. In essence, it is the requirement that any component in a circuit must accept voltages with a certain range for “1” and “0”, and then must output its own results in a certain range. These ranges are defined so as to require that components always produce equal or better conditioned voltages than their inputs, thus ensuring that the digital signals being propagated through the circuit don’t degrade into noise due to lots of tiny losses.

For some reason, I took this concept to heart in a programming sense and have found it is a good rule to code by. My version of the static discipline, adapted for software, is that whenever you are making a modification to a piece of code, you should always leave it in a state of stability equal to or better than how you found it. And preferably the latter.

All too often, people go in to fix a problem or add a feature to a piece of code, and they just hammer on it until it does that one new thing. The resulting code is then usually more fragile, less well designed, more unnecessarily complex, etc. To try to prevent myself from having this effect, I try to observe the static discipline.

And so it was with this Windows key situation. I could see what the code was supposed to be doing, and although I could reasonably fix my bug by just commenting out the hook (because it didn’t work), I felt it was the more disciplined thing to do to figure out why the hook wasn’t working, and to implement something that did what it was trying to do. Since there was no obvious bug, I had to start experimenting.

After adding some debug outputs and running Spy to keep an eye on the window messages, the first thing I tried was removing the part of the code that unregistered the hook when his window lost focus. Since I didn’t know exactly how his windowing code worked at all levels, I figured it wasn’t entirely out of the question that it could be getting disabled prematurely.

Although that turned out not to be the case, this experiment did pay off unexpectedly (as many do): although the keyboard hook still never got called when The Witness window had focus, the hook did start getting called when it lost focus. Yes, oddly enough, the hook was working just fine for everyone else’s window, but on the one window where it actually needed to work, it wasn’t working.

This lead to the obvious question, what was The Witness doing differently?

Raw Input

This was a case where having lots of Windows programming experience probably saved a lot of time. Instead of having to experiment blindly, I knew right off the bat that Windows Raw Input probably had something to do with the situation. I’ve debugged lots of Windows front end code in the past, and any time Raw Input, DirectInput, XInput, or anything else with the word “input” in it is involved, you know that things are going to be going a little haywire.

I commented out the device registration call for Raw Input, and lo and behold, the hook started working for The Witness window, no problem. So, although I was completely unable to find any documentation on MSDN that discussed what was going on under the hood, it must be the case that Raw Input processing happens before low-level keyboard hook processing in such as way as to prevent the keyboard hook processing from ever seeing keyboard messages on a Raw Input-registered window. Normally, this wouldn’t be a problem, but in our case, the Microsoft-recommended fix for the Windows logo key problem relies on keyboard hooks. The Witness needs Raw Input to get good low-level relative mouse movements, but it also needs to disable the Windows logo keys; sadly, this is the kind of situation that is all too routine for anyone who programs Windows professionally.

I couldn’t think of any plausible ways that I could convince Raw Input to start using the Windows keyboard hook, since I don’t have access to any of the Windows source code and I have no idea how it works internally. So I focused my efforts on Raw Input.

I experimented with just about everything I could think of. I tried not calling DefWindowProc on WM_INPUT calls, to see if it was the default window processing that was using the Windows logo key events to switch to the start menu. No such luck. I tried using RIDEV_NOLEGACY during Raw Input initialization to see if preventing keyboard messages from being generated would solve the problem. It didn’t. After many failed attempts, and finding no relevant information on the web, I decided to call it a night and sleep on it.

When I woke the next morning, for some reason I remembered the most important thing anyone programming Windows must remember: never, ever trust the documentation. It’s always either inaccurate or incomplete. So when I sat down at my keyboard, the first thing I did was try using the RIDEV_NOHOTKEYS flag when initializing Raw Input.

You see, the previous day I hadn’t bothered to try this flag, because it explicitly says in the documentation that it does not block system-level hotkeys, only application-level hotkeys (like the kind you create yourself with RegisterHotKey). But why was I believing that? That was just something somebody wrote down when they needed to create a documentation page. They probably never even saw the actual code for Raw Input in the first place.

Surprise surprise, RIDEV_NOHOTKEYS worked. After all the fussing, all that had to be done to make The Witness impervious to Windows Logo keys was one little flag during Raw Input initialization. No keyboard hook necessary.

Zero-Sum Game

It’s worth taking a step back for a moment and asking the question, why did all of this happen? Why did I have to spend several hours working on code that has nothing to do with my project, is not valuable in any intrinsic way, and whose specifics are relevant only in a fleeting temporal sense (there will be no need to know anything about RIDEV_NOHOTKEYS when the next user input model for Windows is invariably introduced)?

The answer is the same on Windows as it is on most every other platform that exists today: the platform providers just don’t spend real resources designing and improving their APIs. If there had simply been an API call, WindowIgnoresWindowsLogoKeys(HWND), then everyone would always just call that, and the platform developers and Microsoft could field it and do the right thing, no matter how many times they revised their platform and its infrastructure.

Better yet, way back when they first started catering to games with Windows 95, they could have introduce a call like BeginFullScreenGame(), and that could have done literally everything for you: initialize the window, set up the input stream, prepare the sound device, etc. Then they wouldn’t have had to ever worry about the Windows logo keys at all, because they would have been automatically disabled in any application that initialized itself with that call.

But instead of any of these things, Microsoft chose to ship a new feature (the Windows logo hotkey) without ever thinking about the consequences. Then they patched over its problems by telling developers to install a keyboard hook (SetWindowsHookEx), something that has nothing to do with the actual problem but which was the most expedient way for them to get the problem solved without doing any real work on their platform. Then they broke that method, which they’d previously advocated, when they introduce Raw Input, so (presumably) they introduced a new way to do the same thing (RIDEV_NOHOTKEYS), although they failed to document it. Then they didn’t even bother to publish an obvious tech article explaining this change, or even revise their old documents saying that SetWindowsHookEx was the preferred solution.

As broken as all that sounds, it is absolutely par for the course. On every project, in every discipline, developers just like me spend hundreds of hours doing nothing but diagnosing and repairing problems with the platforms on which we ship our software. It may sound like it’s just “part of the job”, but the pernicious thing about it is that development resources are always finite. External to the platform, development is always a zero-sum game. The hours I spent tracking down this bug were hours that won’t get spent making parts of The Witness better, and there’s nothing me or the team can do about that. If we want to ship a polished project, it’s a sacrifice we have to make.

Despite not even being finished, The Witness is already one of my all-time favorite games. I’ve been playing it since Jon started developing it several years ago, and I could not be more excited about its upcoming release.

Unlike Jon’s previous game Braid, the asset and programming scale for The Witness is much further toward the “triple-A” side of the scale than the “indie game” side. As anyone who works on these types of projects knows, the work required to successfully pull off a game goes up dramatically when you go that direction. The Witness already has several more people working on it than Braid did at its peak, but as with any project of this size, there are still many parts of the project that need more attention than the principals can afford to give them.

For this reason, it was always my intention to try to find some time to help out on The Witness when it came time to ship it. So this past Thanksgiving, Jon and I sat down and he went through a list of things in the codebase that he thought would benefit from the attention of an extra programmer. After considering the relative importance of the items, we decided that the player movement code was the place where improvements could have the biggest impact on the player’s experience.

Walkmonster in Wall

In the context of The Witness, the goal of player movement code is to be as unobtrusive as possible. The player is supposed to fully immerse themselves in an alternate reality, and every last detail is important to this experience. The last thing you ever want to happen is to have the player notice the fact that they’re sitting at a computer asking it to move a virtual viewpoint around.

For that reason, the player movement code has to be rock solid. If you can get stuck on edges, trapped in walls, fall through the floor, walk through a cliff, walk down a hill but not back up it, etc., it immediately breaks the immersion and reminds you that you are playing an artificial experience mitigated by an unreliable system. It can also lead to disastrous consequences for the player in some circumstances if there’s no way for them to recover from the problem with restarting or reloading a (possibly very old) save game. If you play games often, you’ve probably encountered one or more of these problems in shipping titles, and you know what I’m talking about.

Last week I began to work on this problem. I decided the first thing to do was to write a few integrated tools for working with the player movement code so we could analyze it and see how it was currently behaving. Immediately, when I opened up the project, I found myself with a familiar conundrum: what to name my first source file. This is always the most important part of any project (as Bob Pollard once said of band names and album names). If you have the right name for your source file, it’s smooth sailing from there. Pick the wrong name, and you might as well call the project off.

But what to call a system for bulletproofing player movement code? I’d never been asked to write code for that before. When I thought about it, I realized that I’d only personally seen evidence of similar code once before: when playing an early beta of Quake. There were some bugs with the monster placement, and in the console window, you could see error messages announcing the fact that the monsters, instead of spawning on the ground, had spawned partially intersecting with the level geometry. Each debug message began with the phrase “walkmonster in wall at. . .”

Bingo. It’s hard to ask for a better source file name than “walk_monster.cpp”. I was pretty sure, from that point forward, the code was going to come along nicely.

Drive Toward Point

When you’re trying to test a system, the most important thing to do is actually test the system. Even though that sounds like a straightforward rule, people writing tests often fail to observe it.

In this specific case, there are very easy ways to think that you were testing the player movement code without actually testing it. One example would be to do some kind of analysis on the collision volumes and walkable surfaces in the game, looking for small areas, gaps, etc. Once you’d eliminated all of these, you’d then proclaim the world safe to traverse and move on.

But this is testing the data, not the actual code. It’s still easily possible to have bugs in the movement code that result in bad behavior even with sanitized data.

To avoid this kind of trap, I wanted the testing system to remain as close as possible to what a human does to actually control player movement in the game. I started out by writing two routines to serve as the building blocks of this kind of testing.

The first is the closest to real human usage. It’s an update call that hooks into The Witness’s input processing and feeds it synthetic mouse and keyboard events. It can do some basic things that humans might do, such as looking around, walking toward a point, looking toward a point, and so on. It does these through nothing but emulation of human interaction with the mouse and keyboard, so I know that everything in The Witness’s input path will be running as it actually would during testing. I’ll talk more about this system and how it’s used in later installments.

The second piece of code is one step removed from that level. It’s a function called DriveTowardPoint that takes two points in the world and, calling the existing player collision system, attempts to move cleanly from one to the other. When it returns, it provides information about the attempt such as obstacles encountered and whether or not the destination was successfully reached.

This function is not quite as reliable as the synthetic input method of testing, because it eliminates a portion of the actual player movement system from testing. For example, any erroneous state associated with where the player is that the collision system might have built up won’t affect tests using this function. Nevertheless, I felt it valuable to have this level of testing available because it can much more rapidly test large areas, since it does not require the entire game loop to run, and thus can be employed much more frequently and over the entire world instead of just isolated test runs.

It’s also worth noting that there are no physics inputs to this function; there are no velocities specified for the starting point, for example. This is because The Witness is not an action game, so there are few stateful physical properties for the player. Players can’t jump, they can’t wall-run, they can’t enter bullet-time. Supporting these types of behaviors is possible with systems of the kind I’ll be discussing, but they add a layer of complexity that we don’t have to worry about on this project.

Anyhow, with DriveTowardPoint in place, it was possible for me to start on my first goal for the system: determining everywhere the player can go on the island of The Witness.

Rapidly Exploring Random Trees

Where can the player go? It seems like a simple question, but you’d be surprised how many games ship without the development team knowing the real answer. If possible, I want The Witness to be one of the few games where the team knows prior to shipping exactly where the player can and can’t go, no surprises.

This makes the problem statement, but perhaps not the problem, very simple: given a DriveTowardPoint function that faithfully determines whether a player could move in a straight line between two points, produce a coverage map showing everywhere the player could end up.

For reasons that I can’t necessarily explain, I had in my head, even before writing a line of code, that the best way to do this would be to use a Rapidly Exploring Random Tree. For those of you unfamiliar with this algorithm, it’s a very simple process whereby you record all the points you’ve visited along with a reference to the point from which you walked there. To add points to the tree, you pick a random target point anywhere in the world, select the closest point in your tree thus far, and try to get from that tree point to the target point. Wherever you end up, that’s a new sample point.

Normally, this is more of a pathfinding algorithm, so interleaved with random points, you repeatedly select some goal point as a target. This biases the exploration of the space toward the target, which is what you want when your only aim is to reach the goal. But in this case, I wanted to produce a complete map of everywhere the player could go, so I was exclusively using random samples.

After implementing this (it is thankfully a very simple algorithm to write, so it doesn’t take long), I could see that it did do a reasonable job of exploring the space (the white etchings show paths that have been explored, and the red vertical lines show places where an obstacle was hit):

However, once I actually looked at how it behaved, it was clear to me that it wasn’t really the algorithm I wanted. For example, even after many iterations, it often barely explores rooms like this one, despite densely covering the area just outside, simply because it doesn’t pick enough random targets inside it:

If I’d thought about it before I started, I would have realized that the benefit of something like a Rapidly Exploring Random Tree is that it explores high-dimensional spaces efficiently. In fact that’s the primary reason you normally use it. But in The Witness, we don’t have a high-dimensional space. We have a two-dimensional space (distributed on an intricate manifold, yes, but a two-dimensional space nonetheless).

In this low-dimensional space, the benefits of a Rapidly Exploring Random Tree are muted, and the drawback is critically bad for my stated purpose: it’s designed to most efficiently find ways to connect pairs of points in a space, not to efficiently find all reachable points in that space. If you care about the latter, then Rapidly Exploring Random Trees will actually take an excruciatingly long time to do so.

So, I quickly realized that what I should be looking for is an algorithm that efficiently covers low-dimensional spaces in their entirety.

3D Flood Filling

Once I actually thought about the choice of algorithm, it seemed obvious that what I actually wanted was something like the old 2D flood fills we used to use to fill regions of a bitmap. Given a starting point, I just wanted to fill up the entire space, exhaustively probing every possible way you could go. Unfortunately, this is a lot less straightforward in the world of The Witness than it is in a 2D bitmap, for a number of reasons.

First, there’s no clear concept of finite connectivity around a point. Everything is continuous. It’s not like pixels where you can easily enumerate 4 possible places to go from every given location, and check each one in turn.

Second, there’s no fixed size for a locus in space like there is for a pixel in a bitmap. Walkable surfaces and obstacles can be anywhere, they have no minimum or maximum feature size, and no alignment to any external grid.

Third, although walking around in The Witness locally acts as if it were on a plane, the space itself actually is a deeply interconnected and varying manifold, with walkable regions directly above other regions (stacked perhaps several times over), and special connectivity that changes based on world states (such as doors that open, elevators that go up and down, etc.)

Given these complexities, it is very easy to think you have devised a way to do a flood fill, only to find after you’ve implemented it that it gets bogged down oversampling areas, misses important paths, produces erroneous connectivity information at places where the manifold is complex, or is simply too cumbersome to use because it must be restarted to cope with changes in the world state.

No good solution to all of this immediately came to mind, so I tried some simple experimentation. Using the Rapidly Exploring Random Tree code I had written, I changed the target point selection from random selection to a very controlled selection. Each time a new point was added to the tree, I would say that points a unit step along the cardinal directions from that point where to be considered future targets, just like a basic 2D flood fill.

But of course, this would produce a useless sampling loop if I wasn’t careful. A point would branch out to a neighborhood of 8 points around it, but those 8 points would all then want to try the original point again, and this would go back and forth forever. So in addition to the controlled target selection, I also needed the simple constraint that any target which wasn’t some minimum useful distance away from an existing target point would not be considered. To my surprise, these two rather simple rules did produce a somewhat usable flood fill:

Not bad for just trying the obvious thing. But it suffers from what I’ve taken to calling “boundary echo”. You can see this effect in the following screenshot, taken while the map exploration was still in progress:

In regions with no obstacles, this algorithm proceeds nicely, sampling at a relatively even distance. But once it hits a boundary, the intersections there produce points that are “off grid”, in that they are not aligned along the sampling pattern with which the algorithm filled the adjacent open region. The reason “on grid” points don’t produce overly dense tessellation is because any new point that tries to step back onto one of the previous points finds the previous point there and declines to reconsider it. But when new points are produced on the boundary, they are completely unaligned, so there is nothing to stop them from stepping back into the already explored space. This creates a wave of offset sampling which will continue until it runs into a fortuitous line of points somewhere else that happen to be close enough to its advancing front for it to consider them coincident.

Although this may not seem like a huge problem, it is actually critical. The entire goal of an algorithm like this is to concentrate the samples in the areas where they are most likely to yield productive results. The more time you spend sampling and resampling wide open regions, the less time you spend mapping the actual edges of that region, which is the information you actually wanted. Because this is a continuous space and only an infinite number of samples can truly reveal its exact shape, your ratio of meaningful to meaningless samples is literally the measure of how effective your algorithm is at producing the walkable area.

Now, there is an easy fix for this specific problem: expand the distance at which two points are considered “close enough”. But by doing this to reduce sampling density in places you don’t care about, you lose sampling density in places you do care about, like around boundaries where you are trying to carefully probe for holes.

Localized Directional Sampling

Perhaps because I had started with a Rapidly Exploring Random Tree, my brain had been pigeonholed into thinking about proximity to the exclusion of everything else. The previous algorithms had all been about using proximity to do things, like figure out whether a new point should be considered, or which point to start from when trying to get to a new target point.

But after thinking about the problem for a while, I came to realize is that everything falls more neatly into place if you think in terms of directionality as well as proximity. While obvious in hindsight, if you’ve worked on these sorts of problems before, you know it’s easy to get trapped in one way of thinking for a while and not see the bigger picture, even if it turns out to be a simple one. That is precisely what had happened to me.

Once I shifted perspective, the correct sampling behavior was obvious. Each time I wanted to expand the exploration from a point, I would do a proximity query for a local neighborhood of existing nearby points. Instead of using the distance to those points to guide the exploration, however, I would categorize them by their direction (currently, I just use the eight cardinal directions, but I’d like to experiment with different kernels here).

In any direction where I don’t “see” a point, I walk a predetermined distance and add a point wherever I end up (whether I hit something or not). In any direction where I do see a point, I walk there and make sure I can get to it. If I can, I just add a visible edge, so it’s easy for the user to see that we’re connected. If I can’t, then I do add a new point at the collision, revealing the obstacle boundary.

This sampling method works beautifully. It allows you to very accurately control the sampling with nicely tunable parameters, keeping all the points you want while preventing unnecessary tesselation, which leads to a very fast filling of the space:

Because it searches along directions rather than just using proximity, it is immune to boundary echo, and restricts oversampling to the boundaries where you actually want it:

Furthermore, it’s completely impervious to state changes or complex manifold problems. It deals strictly with points and those points can be anywhere, and you can add new ones at any time. If you’ve already made a map of an area with a door closed, and you open the door, all you have to do is plant a single new exploration point on the other side of the door and say “continue expanding this map”, and it’ll connect itself back up and explore the entire area beyond the door properly.

You can also change any of the core parameters at any time, and everything still works. Want to sample an area more densely? Just set the default spacing lower. You can do this in the middle of a map build, and it will start sampling more densely, with no need to jettison the existing results (which may have taken some time to produce).

Rudimentary Edge Probing

Although the algorithm by default will already sample boundaries more thoroughly, because intersections produce additional points beyond the sampling pattern, it didn’t necessarily probe them as thoroughly as I wanted, because it doesn’t do anything special when it encounters obstacles. It occurred to me that since I know which points were produced from collisions and which were not, whenever two collision points found themselves connected by an edge, I could choose to invoke extra sampling to try to find more boundary points nearby.

I haven’t yet worked on this extensively, but I put in a rudimentary method to test the theory, and it shows promise. For any two collision points connected by an edge, I step to the midpoint of the edge and probe outwards perpendicular to the edge. If I don’t collide with the boundary within a very short distance, I assume that the boundary is more complex, and add a new target there so the search will continue in that area.

Even this simple scheme produces nicely dense sampling along the boundary, without any oversampling of open regions nearby. Here’s a region with several boundaries, but without edge probing:

Now here’s the same region with edge probing:

As pleased as I am with this, I’d be surprised if there weren’t significantly better algorithms for sampling the boundary, and I look forward to trying a few more methods in the future.

Early Victories

Even with little development time, and the resulting rather simplistic code, Walk Monster already produces very usable output that can find real problems in the game. Here are just a few examples of problems I’ve noticed in passing while developing the algorithm:

The slopes on the side of this extended platform should not have been walkable, but they were. This was because the player movement code has a pathologically bad way in which it deals with sloping geometry. I now know that’s in there, and I’ll be sure to fix it when the time comes to bulletproof it.

The Witness is meant to be a contemplative game, but wondering why there seemed to be a rock where there was no rock was not meant to be one of its koans. As you might guess, this was because someone left a collision volume in the game when the geometry it represented had already been removed. This is an easy thing to have happen, and it’s nice to have a tool that trivially reveals these mistakes quickly so people don’t have to worry about it.

These were supposed to be impenetrable rocks, but Walk Monster revealed them to be anything but. What’s worse, Walk Monster revealed that the path was somehow only traversible in one direction (from left to right in this top-down screenshot), which is never supposed to happen. I haven’t actually looked at this problem yet, besides verifying that I could actually do it via player motion (I could). It’ll be interesting to see what’s going on!

Open Questions

It’s encouraging to already have good results upon which we can build. Like I said, if you pick the right name for your source files, it all flows smoothly from there! But all of this work was done in the span of a few days, so it is obviously far from comprehensive and at lot of things are still completely ad hoc. If I have time to develop these systems further, there are a number of questions worth answering.

First, what kinds of post-processing on the data should I do to make it easiest to visualize? Drawing the raw network of points and edges can be very difficult for people to visually untangle, and some sort of better representation might make it possible for people to quickly understand more complex walkable areas at a glance.

Second, how can I improve the sampling patterns around boundaries to ensure that I find the most number of holes? Are there good methods for characterizing the ways in which shapes fall into a lattice, and are there good tessellation schemes that maximize the chances of intersecting and passing through these shapes?

Third, are regular sampling patterns or randomized sampling patterns better for filling spaces? I could easily modify the target point selection criteria to produce randomized patterns, but it’s not clear if this is worthwhile or, if it is, what kind of randomized patterns would be best.

Fourth, what other information would we like to get from our walkable area maps, now that we can build them? For example, it would be a straightforward extension of the existing system to do things like pathfinding or distance mapping, where a user could pick a point and ask for the shortest path between it and some other point, or to see a heat map of the distance between that point and all other points in the map. Are queries like this useful? What other queries might be?

For now, the walkable area visualizations we’re already getting from Walk Monster are more than enough to show us that the player movement code is pretty broken. I had planned to move on to creating a system for overnight testing via the user input method, but it seems clear that we already have enough problems to fix that there’s no need for that yet. So my next order of business, before improving these tools any further, will be to see what I can do to make the player movement code more robust. While I’m at it, I’d also like to see if I can get it an order of magnitude faster or two, because currently, the thing that slows down Walk Monster rather severely is the slow speed of the collision system.

With luck, I’ll report back soon in another installment on how that work is going.

The current puzzle count is 497. (This is lower than last time; last time was an overcount, as we had some temporary duplicates in the world that were being counted mistakenly.)

Right now, we save the thumbnail and the game data as separate files, but the names are mostly the same; so if you go into your game data folder in Windows you can see them living there very plainly:

(It might be interesting to do it like Spore and embed the save data in the image file, so that there’s only one file per save. Maybe we’ll do this, though it’s not a no-brainer; right now the image file is usually bigger than the save data, so this would involve tying a relatively small amount of crucial data to a relatively large amount of cosmetic data. Maybe that doesn’t matter, though.)

As I mentioned in the previous post, we are starting to get the game out to a limited number of players. It’s been a long time since I’ve had anyone new play through the game as a whole, so I really have no idea how it plays now, and it’s important to find out. However, it’s not so easy, because if you sit behind someone and watch them play a game, they are going to play it differently than if you weren’t there. You invalidate your own playtest data.

So, I wanted to set up the game to record some minimal playtest data, that would be easy for me to browse through later on, and that would get recorded without me being there to directly watch the play session.

One of the first approaches that programmers often consider is an input recording method. You just journal a stream of all the player’s inputs, and you can play that back deterministically to reproduce the entire game session. I have never liked this method. Firstly, you need to play back using exactly the same version of the game executable that the player used, which is a pain; secondly, this kind of playback code is brittle and breaks often. An uninitialized memory read bug can render your playback data useless! (Of course, one reason you might want playback data is to track down an uninitialized memory read bug… so… there’s a problem here.)

For these reasons I didn’t want to use input journaling. It seemed to me that our saved games are small enough, and the thumbnails are nice enough, that together they almost solve the problem. All I need to do is save a series of snapshots / game states over time, without overwriting previous saves:

Whereas the earlier folder contained a collection of different play sessions, this one contains one play session at different moments in time. Every time the game made an autosave for this play session (overwriting the old one in the normal way), it also also copied the autosave here. The date and time that are in each filename just indicate when that game was started; the number at the end tells you which snapshot this is.

There’s one immediate, very nice property of this: I don’t even need to run the game in order to get a general idea of how someone’s playtest went. Once they send me the files, I can just click on the first one in Windows to invoke the image viewer, and then click the forward/backward buttons in the image viewer to animate my way through the game.

But, if I have questions about what was going on, I can also dig into the save file that corresponds to any given image. I made a mode that lets me browse these saves in-game, load up any particular one into the current world state, and play from there. I invoke it by running a console command:

This loads all the saved games and thumbnails and pops them into a scrolling list at the top of the screen, in chronological order:

I just scroll to the one I want:

… then press Enter to load that game state:

It’s very fast to use and it only took a few hours to program.

This whole system is also pretty cool because it’s much more robust than the input-recording method. If I want to look into some detail that is super-nitpicky, I may need to use the same build of the game executable that the player used, but this will almost never be necessary. Usually I won’t even need to use the in-game browser — as I mentioned, I can just click through the thumbnails in Windows to get a basic sense of what’s going on.

The fact that we only have discrete snapshots is sort of a side-bonus: because we only snapshot every minute or so, the data is not so creepy and spy-like; players can relax while they are playtesting and know that we won’t later be obsessing over their every move.

Today we shipped an update, still to the same limited group. We have some initial tech issues to work out before we go wider (game packaging issues, graphics card incompatibilities, and the like).

Here’s a desktop screenshot from one of the testers, with someone chiding him about what he’s playing:

(The Witness window in this shot is artificially resized to make room on the desktop; by default we launch at 1280×720.)

I know that the impulse is to think “oh wow they are almost done!” Don’t think that, it’s not the case. We still have a gigantic amount of work to do and it’ll be a while before the game is done.

Random Trivia Q&A

Q: How many footstep sound effects are in The Witness?

A: 1,119 so far. They sound really good! We will probably be in the Guinness Book of World Records as the game with the most footstep sounds…

Turns out this can be implemented easily using some of the new C++ features. I’m not a big fan of C++11; for the most part I think it adds even more complexity to an already complex language. However, I have to admit that some of the features can be useful, when used with care and moderation.

I think this is one of such cases. If you can get past the weird syntax, lambdas can be very powerful. What I find most useful is to be able to define local functions (closures) that can access the local variables of the enclosing scope.

In our case, what we want to do is to define some code that is executed at the end of the scope to cleanup objects allocated in the current scope. This is easily achieved allocating an object on the stack that invokes a lambda function that wraps our custom cleanup code:

#include <functional>

struct ScopeExit {
    ScopeExit(std::function f) : f(f) {}
    ~ScopeExit() { f(); }
    std::function f;
};

#define STRING_JOIN2(arg1, arg2) DO_STRING_JOIN2(arg1, arg2)
#define DO_STRING_JOIN2(arg1, arg2) arg1 ## arg2
#define SCOPE_EXIT(code) \
    ScopeExit STRING_JOIN2(scope_exit_, __LINE__)([=](){code;})

Then you can write code like:

int * ip = new int[16];
SCOPE_EXIT(delete [] ip);

FILE * fp = fopen("test.out", "wb");
SCOPE_EXIT(fclose(fp));

I know you can go to greater lengths to mimic the D syntax, and that Boost also provides similar functionality, but this does all we need in less than 10 lines of code.

Sadly we cannot really use this on The Witness, because I’m not sure we are yet ready to drop support for platforms and compilers that do not implement the required C++11 functionality.

Update:

As mentioned by Daniel in the comments, you don’t really want to use std::function in this case. A quick look at the assembly output confirms that the generated code is horrible. It makes a lot more sense to wrap the closure explicitly and rely on type inference to allocate the wrapper object on the stack:

template <typename F>
struct ScopeExit {
    ScopeExit(F f) : f(f) {}
    ~ScopeExit() { f(); }
    F f;
};

template <typename F>
ScopeExit<F> MakeScopeExit(F f) {
    return ScopeExit<F>(f);
};

#define SCOPE_EXIT(code) \
    auto STRING_JOIN2(scope_exit_, __LINE__) = MakeScopeExit([=](){code;})

Current puzzle count: 528.

They may not look tremendously different from this off-island vantage point, but when you are down there moving around, the difference is substantial. I feel a lot better about the layout now.

One of my primary design goals, all through development, has been to keep the island spatially small, so that there isn’t a lot of wandering through empty or pointless spaces. The idea is to take the best parts of an open-world game, involving choice, exploration and discovery, but compressing these into a maximally dense experience.

That has been going well, but there’s an interesting phenomenon where, as we add more detail to the world, it starts to feel a lot bigger. When something is a mostly-featureless plain it is easy to feel like it’s small, but once you are passing tiny rocks and plants that are set up very carefully, your brain has more scale reference. So there have been a couple of times during development where we stopped and said “Hey wait a minute, things are just too big.”

Last week that happened again; we were looking at two different parts of the island on the same day, and both of them seemed much bigger than we needed. So, we decided to go back to the overall island plan and try to compress it further. We had a discussion about this on Thursday, then the architects talked it over; you see the result of those discussions in the image above. These maps are a handy guide to helping us figure out what we can cut and how we can scoot existing areas around to pack them a little better, and to better-define the areas between them.

This is a little painful, since we almost had the island back into playable shape, and this knocks it back out of playable shape for a little longer. But my hope is that we can just make a mess and then clean it up very quickly.

In other news, there had been a puzzle that I needed to design for two years — in the sense that I knew I needed to build the puzzle in order for a certain area to be complete, and I knew what that puzzle vaguely had to be about, but didn’t have any good ideas on how to build it. Last week I had a breakthrough on that and built it very quickly; it’s now one of the best puzzles in the game. So things are coming along well!

The colors are all funny because we haven’t written the shaders for the iPad yet, so what you are seeing is the surface normals of the objects in the scene.

We haven’t written the shaders yet because they need to be OpenGL ES shaders, and the shaders we have right now are HLSL shaders all written for medium-to-high-end PCs. As the first part of the porting work, Andy made the engine work on OpenGL on the PC, and for that we converted our HLSL shaders to GLSL automatically. But the jump to ES on an embedded device is big enough that it doesn’t make sense to try and use the same shaders. Instead we’ll have to have some simpler shaders that we substitute.

As you can see from my previous post, I have been working on touch controls (not on the iPad, but on a touchscreen Windows PC). A very rough first cut of that is now working. It’s nice to play the game with touch controls, as it feels more immediate than mouse and keyboard (and I have been a first-person shooter player forever!)

So in order to have the game fully interactive on the iPad, we just need to write the bit that translates iPad touch controls to feed them into the rest of the gameplay code, which should not take long.

Then, after that, we have to work out exactly how we are going to crunch all the graphical assets down to portable versions and do the streaming in an appropriate way… which is going to be a bit of work. Salvador built a streaming asset system a while back, and we have been using that daily on the PC, but we haven’t tried to set it up on a lighter-weight portable system like the iPad yet. We’ll see how it goes!

I am not sure that many people are going to use Windows for touch devices, at least in the Windows 8 timeframe. But at least some people will, so it’d be nice to have a high-quality touch interface built into the game on Windows.

With all this in mind, I started working on some Windows 8 touch hardware a couple of days ago. There is a message format that you can use in Windows 8 to get events from the touch hardware, WM_POINTER, but because I want the touch interface to work in Windows 7 as well, I wanted to use WM_TOUCH messages, which were introduced in Windows 7 and are supported in both operating systems.

It look me less than 20 minutes to get WM_POINTER working. It took many, many hours to get WM_TOUCH working, because WM_TOUCH is some kind of fully insane mess. None of the insanity seems to be documented, or discussed on the internet, so I am laying it out here in case anyone in the future stumbles into this same trap.

A WM_TOUCH message will never be returned by PeekMessage. Spy++ will tell you that WM_TOUCH is being sent to your application, but before the message is delivered, it gets stealthily converted into a mouse motion event. The type of this event will vary depending on whether you have raw input selected: if raw input, it will be WM_INPUT (0x00ff); if not, it will be WM_MOUSEMOTION (0×0200).

If your application handles these events, you may think it’s reasonable to complete the handling without calling DefWindowProc. If you do this, you will never see a WM_TOUCH, because something buried down in DefWindowProc stealthily dispatches a WM_TOUCH to your application’s windows when it sees a specially-marked WM_INPUT or WM_MOUSEMOVE.

Note that actually, you are supposed to always call DefWindowProc on WM_INPUT (for “cleanup” — I don’t know what this means and why they can’t just clean it up when you get the next message — but whatever); but WM_MOUSEMOVE has no such requirement, and if you believe you handled the event, it is natural *not* to call DefWindowProc. This will result in your never seeing WM_TOUCH. Strictly speaking our game had a bug, in that we were not calling DefWindowProc after WM_INPUT, but the bug was hard to find because it persisted when I tried turning off raw input, because we also were not calling DefWindowProc after WM_MOUSEMOVE (which as I mentioned is, in theory, totally fine.)

Important to note: WM_TOUCH gets dispatched to your windows without you dispatching it! You may think that you can log all messages after they come out of PeekMessage, and that you are in control of your entire message stream, but this is apparently not true. WM_TOUCH will just teleport itself into your window.

In addition to the mouse motion events, your application will get a bunch of other mouse events that are generated by the touch interface, to emulate mouse stuff for programs that don’t understand touch. There is no way to tell Windows you don’t want these events (unless you do Windows 8 stuff and respond to WM_POINTER; see below). According to Microsoft, the official way you are supposed to detect and ignore these messages is like this:

#define MOUSEEVENTF_FROMTOUCH 0xFF515700
if ((GetMessageExtraInfo() & MOUSEEVENTF_FROMTOUCH) 
        == MOUSEEVENTF_FROMTOUCH) { 
     // Skip message
}

I wish I were joking. It’s been this way for years; Casey wrote about this in 2008.

This mask works for events like WM_LBUTTONDOWN, but it does not work for the mouse motion events e.g. WM_INPUT. If you want to take input both from touch and the mouse, you need to disambiguate the real WM_INPUTs from the fake ones. Through a process of experimentation with message flags, I landed on this:


if ((GetMessageExtraInfo() & 0x82) == 0x82) { /* ignore event */ }


It seems to work for now, but who knows if that will mysteriously break in some future version of windows.

Nicely, if your application handles WM_POINTER messages, you will not get any of this mouse emulation stuff or WM_TOUCH, so all this ugliness disappears. If it is okay to make Windows 8 your minimum specification, I recommend using WM_POINTER and staying far away from WM_TOUCH.

I ended up running a straightforward test where we would change the game minimally and build consecutive patches. The patches turned out to be quite large, but upon investigation, it was clear there usually wasn’t a good reason for this. Our compiled shaders would contain small differences, as would many meshes in the game.

These looked like uninitialized memory errors and, upon further investigation, that’s indeed what they were. They are fixed now, so patches are a lot smaller.

However, patches may never be as small as we would like, and I find there is a pretty deep philosophical issue where.

Right now we have 17MB of files that change in response to minimal changes in the game (say, moving a few entities in the world by small distances). These are mostly cube maps for Light Probe entities, which are used to do lighting on nearby objects. Here’s what a Light Probe looks like in the game:

(Well, you don’t see that sphere the game, only in the editor!) To generate one of these cube maps, we render the scene in all directions, from the point at the center of that sphere, then apply a blur to the result.

Imagine that we move a couple of entities in the world, even slightly. All Light Probes that can see those entities are going to change, by at least a pixel or two. Because we are applying a big fat blur kernel, a 1-2 pixel change becomes small changes spread across the entire image.

We have an automated script that preps the game for release, clearing out data that we only use for development, and ensuring that automatically-computed things get recomputed (like LODs for faraway objects). Recently we added a step to this process that re-generates all the Light Probes, because it just makes sense to update those automatically. If we have to keep track of them manually somehow we are just going to make mistakes.

What this means is that you can pretty much guarantee that a lot of light probes are going to change if you move some entities and patch the game. If we change anything basic about any of the shaders, which seems very likely, then all of the light probes are going to change (because the output of the shaders feeds back into the light probes).

That said, maybe it’s the case now that more light probes are changing than I would expect. I am wondering if other things in the game are causing them to change. For example, when you are playing the game, foliage waves in the wind. I wonder if the animation timer for plants is ticking during the process when we generate light probes, causing them to move over time? This would cause most light probes to be different every time we bake them, regardless of other factors.

Now, there’s a bigger issue. In addition to light probes, we have pre-baked lightmaps all over all geometry in the game:

In a way that is reminiscent of the light probes (but a bit different), we generate these by placing a camera at many points along each surface, rendering the scene from each camera, and interpolating those into a final result.

So, just like light probes change when entities move slightly, lightmaps are going to change also. Lightmaps will also change if shaders change, but furthermore, lightmaps will change if light probes change (because light probes determine the look of entities that shine on the surfaces we are lightmapping), and light probes will change if lightmaps change (because the lightmaps influence the objects rendered in the light probes). So we have a circular dependency, which is not so bad in practice, except it means that if you change one entity in the game, then repeat this process iteratively, you may well end up changing most lightmaps and light probes in the game.

The light probes are not so big, but last I checked there are between 500MB and 700MB of lightmaps. However, re-baking the lightmaps is not part of the automated build process; we have to trigger a bake manually. But it seems like good policy to do a full rebake any time we change the game significantly. So, we are probably looking at a minimum patch size in the 500MB range.

I guess that is not such a big deal, since we are not an online game that would be patching all the time. Hopefully once we release the initial version of the game, the “content” won’t change very much (sorry, but the word “content” as used in video games is kind of absurd, so it’s hard for me to use it without sarcasm quotes; it’s almost as bad as “title”, which is a word I refuse to use, except to make fun of its use)… the “content” won’t change very much, so patches would be rare.

We’ll see. We have a while to figure these things out.

September 2011:

August 2012:

You can see Ignacio’s new water in this shot (which will continue to improve, and already we have a lot of control over water visuals that we are not using yet). There’s a bunch of new audio in the game. Art folks are working on revising various areas and getting them into fully playable shape. Lately I have been thinking hard about how to improve various puzzles (a couple of puzzles already just got *way* better this week), and I have also started rearranging the jungle area, which you see in the left foreground. We’ve had a lot of discussions with the landscape architects, and they’ve given us a detailed topography for the island that helps define some of the features we think will be good to have, like this river you can see to the left of the mountain (solid blue for now; it’s temporary!).

Also, a couple of us have been on vacation!

From: Ignacio Castaño
Date: 24 August 2012

I just checked in a few changes to the water shader. Note that this is still work in progress, there are some artifacts and unfinished features.

From the art point of view, the main change is that there’s now a water_color_map and a water_flow_map that you can use to change the behavior of the shader.

The RGB channels of the color map control the water extinction color and the alpha controls the opacity factor. The previous underwater shading model was simply wrong, the new one works nicely and is easy to tweak. Note that I assume the water color changes very smoothly, avoid sharp color changes!

The flow_map is intended to contain additional water attributes. Right now it’s only used for the stillness, but I will certainly add more in the future.

You will also notice that the shoreline now also has a foam layer:

The way it’s computed is by comparing the screenspace depth difference. This is not very correct, it looks OK from some angles, but it breaks badly when the underwater geometry has screen-space depth discontinuities:

If the foam is a desirable feature, I’m considering modifying the print_map command to render a foam map, or maybe this is something that can be painted manually and added to the water_flow_map. One of my concerns is that this map may not have enough resolution.

Just Cause 2 seems to use this method to render foam, but they apply a big screenspace distortion to hide the artifacts, I found it sickening when you stare at it closely…

The way the water is shaded has also changed somewhat. First, I now approximate the specular exponent based on the position of the camera with respect to the water and the water roughness. Somewhat like the LEAN paper does, but assuming the roughness of the water is the same for the whole texture. Depending on the value of the exponent I either sample the mirrored reflection map, a filtered environment map, or blend between the two. My goal was to be able to use the mirrored reflection map only on a small subset of the lake entities. The physically based approach does not allow us to cull many, so I’ll probably have to introduce some hacks. BTW, I’m doing all these work not just an optimization, but also so that we can use the light probe reflection model in the river and lakes inside the island, where the mirrored reflection map is not available.

The main problem with the filtered environment map is that it does not take local reflections and occlusion from the environment into account. I have two ideas to work around this, one is to store an ambient occlusion map in the lightmaps, this may be a bit too smooth and will lack directional information. The other is to do screenspace raycasting of the depth buffer to decide whether the environment map is occluded or not.

This is somewhat like what Crysis 2 does. Insomniac also uses a similar approach with just a single sample:

http://www.insomniacgames.com/tech/articles/0409/files/water.pdf

I’m hoping we can get away with a crude approximation too. I did a quick test, where the maths are not exactly right, with only 4 samples and no smoothing, and it was starting to look promising:

This won’t allow us to use the filtered environment maps closer to the camera, but if the screenspace raycasting works well, I’m thinking about using non-filtered environment maps in place of the mirrored reflection map, whenever those are not available.

Finally, I’ve also been playing with Valve’s water flow model, I have the basic implementation working, but I haven’t yet been able to reproduce the workaround for the pulsing and repetition artifacts that Alex proposes.

http://www.valvesoftware.com/publications/2010/siggraph2010_vlachos_waterflow.pdf

A couple of months ago I was playing Super Monday Night Combat; a much-anticipated weekly update came along, but its file size was over 800MB! MNC is an Unreal Engine game and Unreal, like many game engines (and like The Witness), packages its data files together for robustness and speed. This package format interacts with the patch delivery system of any online distribution service, though.

Steam’s system performs a binary-data comparison between the old version of your file and the new version, sending only the pieces that have changed. To make this analysis tractable, files are thought of as collections of 1 megabyte chunks; if anything in a particular chunk changes, even one byte, the whole chunk gets included in the patch. If you aren’t careful about this, small details of your package format can trigger very large patches, and this is what was happening with Super MNC (not really the developers’ fault since they were using a licensed engine; I have heard that the Unreal guys are addressing the problem.)

The Witness packs most of its data into a single 2-gigabyte file stored in the .zip format. Before uploading the initial version to Steam, I took a safeguard to minimize patch sizes, or so I thought: Files are sorted lexicographically and then placed into the zip file in that specific order. (Imagine that files are stored in random order: probably every patch involves every player re-downloading the entire game!)

Last night, as a test, I pushed a new build to Steam. The hope was that the patch would be small, since we had only done two days of work. Instead, here’s what we got:

The patch was 660 megabytes. Clearly I had not done enough to prevent this. I opened up a zip file reader and took a look at the file order. In many cases, it was as you’d expect, but in the case of meshes and lightmaps we see this:

‘save’ is the world name and the 5-digit number is the identifier for each entity. (The world name is redundant right now, since The Witness takes place entirely in one world). We re-bake lightmaps pretty often, but we change meshes much less often. So, the packing order you see here will result in us patching more chunks than we should. Not necessarily by much, in this case, since meshes are much smaller than lightmaps, but that might not be true for all meshes, and other data types may have a similar issue but exhibit more-severe problems. So, I had the thought that instead of sorting things like this:

a.lightmap
a.mesh
b.lightmap
b.mesh


What we really want is to sort by file extension first, then sort by name:

a.lightmap
b.lightmap
a.mesh
b.mesh


… but not always. Sometimes we have multiple files associated with one source file, and that will all change when that source file changes, but that have different extensions. This is the case with shaders:

So I hacked together a solution that sorts anything with an extension of .shader*, in terms of the global file list, as though the extension were only “.shader” — that means all these files will stick together — but locally, you still need to keep the full extension, so that the files for one particular shader do not randomize their order every time you make a patch.

The way I do this is to tack a modified extension onto the front of every filename, for sorting purposes only. So “hello.texture” becomes “.texture hello.texture”, “whatever.shader_ps_text” becomes “.shader whatever.shader_ps_text”, “whatever.shader_dump” becomes “.shader whatever.shader_dump”.

If this doesn’t make sense to you, don’t worry too much about it, because it’s not that big of a problem (and I am not even sure if those ps_text and vs_text files are needed for the running game; if we should be discarding them it would certainly mitigate the issue.) These kinds of problems would modestly increase the patch size, but I would not expect the increase to be terrible.

The whole time I was doing this sorting stuff, I knew it wasn’t the real problem, and in the back of my mind I was thinking, “I hope the Steam patch mechanism ignores timestamps on files so that we don’t have to go touching all files to a known time. (Or I hope we aren’t packing a newly-generated build ID into each file or something.)” But this was an example of my brain not working properly: because we are packing all this stuff into a zip file, Steam doesn’t know they are separate files! Since the zip file stores timestamps for all the files, of course that data changes every build (since our build process generates new files every time).

At first I was thinking that this wouldn’t be the real problem, either, because the zip file format has a main directory that is stored compactly, and that is where timestamps would be, because you want to be able to access that data quickly. So I would have guessed that, if all the timestamps changed, the worst that would happen is that the directory would change every build, but that is at most a couple of megabytes. But looking at the zip file specification, I discovered how wrong I was. Inexplicably, timestamps for each file in a zip archive are stored both in the central directory and in the local header for each file, so if you change timestamps for all your files, you are strewing changes everywhere.

So I made the packaging script set all timestamps to 0, but the zip library we are using didn’t like that, so I set the timestamp to January 1, 2012 00:00:00 GMT.

After fixing this, as a test I built two patches in a row from scratch, with the game recompiling all shaders and re-compressing all textures each time. In theory the results should be exactly the same (certainly if this still generated a huge patch, we would still have some kind of big problem). Happily, we have found at least some bit of success:

I am not sure what it is saying there about 2046 of 2262 matching, or 60 changed chunks, but saying it uploaded 0 chunks. In any case it’s better than before. I am going to ask the Steam people what these various numbers actually mean.

Feeling good about this, I ran a final test where I moved around a few objects in the world and changed a constant in one shader:

That’s once again not so good (140 MB!) I had been sweeping the 18MB from the previous run under the rug, thinking “hey, it says 0 chunks sent, so maybe that 18MB is protocol overhead involved in figuring out which chunks changed, and it doesn’t impact patch size.” Apparently not. The next step involves asking the Steam folks for advice, hunting around for a binary diff file with which to analyze our own packages to see if we are doing something unsavory, etc.

Here’s the current Perl code for the part of our script that builds the zip file:

sub modify_name {
    my $name = $_[0];

    my ($file, $dir, $ext) = fileparse(lc($name), qr/\.[^.]*/);
    my $orig_ext = $ext;

    if ($ext =~ m/^\.shader/) { 
        $ext = '.shader';
    }

    # Throw away 'dir' since we discard it when writing the file, anyway.
    return "$ext $file$orig_ext";
}

sub make_uncompressed_zip {  # Sort zip members alphabetically so that binary diffing will have an easier time.
    my ($source, $prefix, $dest_zip_file) = @_;

    my @unsorted = glob("$source/*.*");
    my @sorted = sort { 
        modify_name($a) cmp modify_name($b)
    } @unsorted;

    print("Sorted $#sorted files.\n");

    use Archive::Zip qw(:ERROR_CODES);
    my $zip = Archive::Zip->new();

    
    # Zip API complains if time is before 1980, so use an arbitrary timestamp in that range (yuck).
    my $timestamp_2012_jan1 = 1325376000;
    foreach my $fullname (@sorted) {
        my ($file, $dir, $ext) = fileparse($fullname, qr/\.[^.]*/);

        # print("Adding '$fullname' as '$prefix$file$ext'\n");
        my $member = $zip->addFile("$fullname", "$prefix$file$ext");
        $member->setLastModFileDateTimeFromUnix($timestamp_2012_jan1);  # Clobber timestamp. 
    }

    # Set the compression level on all the members.

    my @members = $zip->members();
    foreach my $member (@members) {
        $member->desiredCompressionMethod(COMPRESSION_STORED);
    }

    $zip->writeToFileNamed($dest_zip_file);
}

We finally got that working well enough; here’s me getting ready to download the game on Steam:

And here I am after having launched the game:

I don’t want people out there to get too excited about these screenshots: this does not mean the game is close to release. You wouldn’t want to play it right now. But this will help us a lot when it comes to getting the game out to playtesters and keeping it updated.

For actually shipping our game off to Steam, we are using the new version of the Steam developer tools, and they are really good. The old stuff (which I had used for Braid) was serviceable but had a lot of undesirable things going on. The new stuff is exactly what you want: you write a couple of very small and easy to read config files, then you run a command and it uploads your game. This process is now fully-automated, which means developers can modify their game and push changes out to users faster and with less hassle than with any other online distribution service. (It is basically as fast as if you host the game yourself).

Lately, Ignacio and Salvador took the features all the way so that we can use them easily in normal workflow. (This happened over some period of time, but the last parts came together this week). Now you can take a screenshot, screw around with the colors in Photoshop, and load it back into the game to control the grading. I am pretty sure this is not our idea, and we got it from the Uncharted guys or someone like that (Update: The idea was developed by Naty Hoffman and others; see the comments to this article).

Here’s how it goes. First, I find an area the look of which I would like to change:

Next I hold down the control key while taking the screenshot, causing it to be saved with a special color bar along the top of the image:

Then I go into Photoshop and screw around with the colors. Here I am using the Hue/Saturation tool to shift the palette away from red:

Or, instead, I may wish to shape the histogram of the image:

That color bar at the top of the screen acts as a look-up table. The game doesn’t need to understand what operations we have performed in Photoshop; it just knows what that color bar looked like when the screenshot was saved out, and, by comparing it to the colors in the result, knows how to transform any color in the rendered scene.

In the game, you can now attach these grading operations to spatial markers, so that color grading kicks in when you are inside specific areas. Here I am setting up some markers in the editor (the marker is the big white box):

After setting up two of these boxes, one for each of the two weird color operations I created, then I can walk around in the game and have these operations apply to everything:

This gives us a really nice degree of control over how the game looks. We plan to use it to help with the color theming that will tie together specific areas.

As you’ll notice, we are starting to work on that big mountain that is closest to the camera. (Finally!) This is still an early concept; it will change a lot.

Ignacio recently implemented a clean way to print an overhead map of the island, in flat colors, which is probably useful for marking things up:

On the modeling side, Eric has been refining an area that’s far from the camera in these shots (you’ll see some screenshots of the interior soon). Luis has been working on the style of our outdoor scenes; we have interesting problems that involve maintaining a visual style that is good for gameplay while also creating images that are most striking. Orsi has been working on refinements to a few particular areas, particularly, adding some deeper puzzle complexity to the area with the tall trees (off to the right in the screenshot image).

Until recently I had been bothered by some unanswered questions regarding the endgame. There were things I wanted to do that didn’t fit together and that felt clumsy. But thankfully, after taking a relaxing weekend where I barely worked on the game at all, on Monday solutions to most of these problems just popped into my head. So that’s cool.

There are still some questions about the gameplay but I think the biggest ones are now nailed down. Over the next couple of weeks I will be implementing the remaining parts of this endgame.

Tech-wise, Andy is pretty far along with the OpenGL renderer, which will help us hit other platforms, and Salvador has tied up the basics of asset streaming and is doing some rendering optimization stuff. Ignacio is refining the LOD system in addition to many miscellaneous tasks.

The original letter went through a few examples of specific issues of certification and how they could be handled better, to help substantiate why I think these processes are so broken. These are not the kinds of details you would expect to make it into a short article for a general audience, but having typed them up, I think some people out there might be interested (especially developers who have not done console games before but are thinking about it). So, below is my original letter (edited a little bit for clarity):

—–

Hi Kyle,

I have gone on record a number of times, saying that the XBLA certification process makes games worse rather than better. But that is a pretty old experience for me now (4 years ago), and they are unlikely to change any of that stuff until the new console comes out anyway. Keep in mind that the Xbox people as a whole do not think of XBLA as a high priority (you can get a glimpse of this from the E3 press conferences, etc).

But, I think the more important issue applies globally, to all of Microsoft, Sony and Nintendo (and not just to smaller downloadable games, but to every game regardless of size). The certification processes of all these platform holders were based on the idea that all these steps they test are absolutely necessary for software to run robustly, and that software robustness is super-important for the health of their platform and its perception by customers.

But, look at iOS. There is almost no certification process for iOS, so by the Microsoft/Sony/Nintendo theory, the apps should be crashing all the time, everyone should think of iOS as sucky, etc. But in fact this is not what is happening. There is no public outcry for more testing and robustness of iOS software.

Part of this may be that iOS software is so easily patched; so maybe a heavy cert process made sense back in the disc-only days, but as we go into the next console generation it becomes unnecessary. But something tells me that Microsoft/Sony/Nintendo are not really going to let up on cert, even though they say they will. It’s just “not in their DNA” as they say. The proof of this is the bizarre over-complexification that already happens with console games today, and is baked into the current certifications, that any of them could easily fix, but none of them do, because they don’t care.

For example: every single game is REQUIRED to say on startup, “sometimes this game saves, when you see this animated icon in the corner, DO NOT TURN OFF YOUR CONSOLE, etc”. This is something that developers have to implement and that has to be tested, which costs significant time and money, but worse than all that, it impacts the user experience, because the startup of the game becomes just a little more bureaucratized, and also — this is supposed to be a fun experience, so why are you issuing warnings and strict instructions? (Just one thing like this may not seem like too much, but combined with everything else, it is a lot; I am using it as just one specific example).

Why is this DO NOT TURN OFF YOUR CONSOLE warning there? It’s because if the game is saving a save file, overwriting an old one, and you turn off in the middle, you might corrupt the saved game. Well, guess what… the solution to this is to IMPLEMENT A MORE ROBUST SAVE SYSTEM. You save the new file next to the old file, flush the I/O, and only delete the old file once the integrity of the new one has been verified (or just don’t remove it, and keep two copies of the save at all times). On load you just make sure you load the intact save file if one is corrupted. This is not hard to implement; I did it in Braid. But if consoles cared about this kind of thing, it would be built into their basic save-file API, so that it would always work perfectly and no developers would ever have to think about it.

If they did this, there would be fewer things to certify, certification would cost a little less and take a bit less time. Let’s say you save 3 days of development and testing per game (this is conservative; the real amount can be substantially higher when you factor in the discussions and coordination about how the save notice should look, etc). Now add up how many games have been released just on the Xbox 360, multiply that number by 3 days, and what you get is probably OVER A DECADE OF DEVELOPER TIME that was wasted. Just on this one little requirement. For something that should just be built into the system by one person in a couple of weeks. Okay, that was only the Xbox, so now add in all the PS3 and Wii games, and see what a huge waste of time that represents.

Like I said, just this one thing by itself would not be too bad, but it is only an example. Another example that all three certification processes have is the “title-safe region” restriction. This basically means that you are not allowed to draw text or gameplay-critical information near the edge of the screen (about 10% of screen width in a margin around the edge) because you don’t know if the user’s TV is displaying the whole picture (even modern HDTVs often cover the edge of the display with a bezel or else just crop it entirely, which is completely stupid).

Okay, so at first blush this seems to make sense, so to make sure players can see everything, we have to go through each screen and each mode in every game we ever release and certify that important things don’t get drawn too far toward the edge. Well, as you can extrapolate from my previous example, this also has wasted DECADES OF DEVELOPER TIME (probably many decades, because dealing with title-safe probably takes a lot more time than save/load). And it is also something that these consoles could handle at the system level if they actually cared about the problem. Just put a screen calibration menu in the dashboard, and then initialize it properly so that players would almost never need to use it (when they are using HDMI output, initialize the settings using the display’s EDID; if they are outputting a different way, just default to shrinking the display a little bit).

10% at the edges of the screen may not sound like much, but it is actually huge. Suppose a game is rendering at 1280×720… you are only really able to use the interior 1024×576, with everything outside that just being decoration. 1280×720 is 921600 pixels; 1024×576 is 589824 pixels, in other words, you only get to use 64% of the screen area for real! Everything outside that is only allowed to be decorative. You know how the 360′s new Metro interface seems to use an absurdly small amount of real estate in the middle of the screen? That is due in large part to title-safe region. You know how games always seem to have their HUD elements placed too far inward when they could be nicer, further off to the sides? Same.

This would also have the benefit of saving work for most people who implement 2D games. In Braid we had to program a screen calibration menu. I notice there’s one at the beginning of Dyad, and one at the beginning of PixelJunk Shooter. For God’s sake why??

This makes all games worse all the time, and makes developers have to do a lot more work, but ultimately could be fixed very easily by any of the platform holders. They just don’t care enough to do it. I think they will have a hard time getting rid of these rules because bureaucracy is in their DNA.

These are just two examples of many… I am going to stop ranting here, but this is just the tip of the iceberg. Most certification requirements are like this.

The edge that both Apple and Valve have going into the future is that they both genuinely care about the end-user experience and want to make it as good as possible. (Their end-user experience is already way better than any of the consoles and they are always working to improve it). Which coincidentally seems to be the place that these consoles are handicapped due to their corporate culture. Can anyone look at the current 360 or PS3 dashboards and legitimately say that those are products of an entity that deeply cares about user experience?

A question a lot of developers have is: When the 720 / PS4 get launched, how many people are really going to care and buy games for those systems? Obviously some people will, but if it is less than bought games this generation — which a lot of developers think is very possible — then look for a “peak oil” kind of crash where a lot of too-big publishers and developers fight over a shrinking market. If the actual way the next-gen consoles work is much like now, they will be a functionally archaic in the marketplace (keep in mind that they have to compete with the iPad 4, 5, 6, 7, 8, and 9. Have you got any idea what the iPad 5 or 6 are going to look like, how powerful they are going to be, what other user experience benefits they are going to have? I sure don’t.)

—–

A few days after having written this letter, I just have one more thought to add: I know that Microsoft thinks they are fixing these issues with their next console, and creating an environment more suitable to free-to-play games and downloadable things in general; I assume Sony is thinking similarly. I will be very surprised if either of them succeeds in fixing these problems. Large institutions always undercorrect; they always think they are being radical and risky when in fact they are doing something tame that is just a modified version of the status quo. When large institutions need to change course by 95% in order to do well, even if they know it’s an emergency and are totally panicked, they can probably only manage about 35%. For recent illustrations of this, look at Nokia and RIM (or, uhh, look at the large governments of Earth in dealing with global warming).

So I would bet that Microsoft and Sony believe they are streamlining their online stuff, because they are implementing an expedited, simplified cert process for patches and content updates, or something, and that because of this they will be in a great position to compete with Facebook and Apple and Steam and whoever. Whatever they do is very likely not to be enough. Their competitors are not stopping either. (Steam, which was already pretty painless in terms of updating games, recently revamped their system; the new thing is way better than the old thing, which was already way better than what the consoles do.)

Originally I wasn’t sure whether I wanted to implement subtitles or not. It’s important to me that the game have an overall aesthetic that contains no visual language. I was worried that if we were to draw subtitles for people who don’t speak English, it might ruin this. But, a year ago, I was talking to a friend who has experience in film, and he convinced me that people just understand that subtitles are a different thing, that they are a layer on top of the game or movie or whatever.

So, I am implementing this, and maybe it’s good, because if I had been hard-line about things, it would mean that most people in the world would not be able to understand the recordings. That’s not terrible, because they are not necessary to complete the game, but as long as I don’t feel like, ultimately, the subtitles are ruining things, it is probably better to have them.

(It’s going to be an interesting time when I go try to implement the main menu without any written words…)

It’s been a very productive day. I implemented the basic subtitling system in less than 3 hours (making a subtitle data file for one of the voice tracks, loading it, detecting in-game when a voice track is playing, finding the appropriate piece of subtitle for the current time, and drawing it on the screen). Here’s the result:

The data file is a very simple format that I just made up. Taking my experience in making Braid multi-language, and looking at the kind of text we will be dealing with in The Witness, it was clear that I needed a format that was mostly about the text, with very minimal markup. Right now the main markup we need is just “which recording is the following text for” and “what time in the recording should the following lines be displayed”. I can imagine that later on we will add some more features to it, but probably not many.

Since The Witness inherits code from Braid, we already have handling for other languages in our font rendering, so I pumped the text through Google Translate to test a couple of other languages:

If you are playing in English, subtitles are OFF BY DEFAULT. It amazes me how badly many games handle subtitles, most notably, that a great many games seem to have voice acting in English and then also default to subtitling in English. This is terrible because people read at a different speed than they listen, and they probably read faster. It ends up dividing peoples’ attention in a weird way. I don’t know why so many games do this except that they aren’t paying enough attention to user experience. So to have a good experience you have to go turn subtitles off, but sometimes that doesn’t even work! A couple of days ago, I was playing Xenoblade Chronicles, and when I turn off subtitles, only about half of the subtitles in the game go away. Apparently there are two different classes of subtitles and one of them you cannot even turn off! I think they never even tested it…

The current puzzle count is 415 (I think we have been over-counting a little recently; this is a more-accurate count).

In case you missed it, I did a live Q&A with Chris Hecker of Spy Party today on Kotaku.

In terms of world-building, lately Luis, Shannon, Eric and Orsi been doing a few things: (1) revisiting the most complicated areas in the game and making them playable with something resembling shipping art assets [until now they were playable only with placeholder pieces]; (2) starting to figure out what the game will look like, outdoor-wise; (3) stuff I can’t spoil in a post like this.

On the tech side Ignacio is working on optimizing the game, as often. A big part of this has been working on the mesh LOD system, but he’s also looking at performance issues related to specific subsystems like grass rendering. Salvador has done a bunch of work on asset streaming, which seems to be pretty solid so far. Andy made a number of improvements to the collision system and audio system, and has now started work on an OpenGL port of the rendering, which, when done, is a big foothold toward portability to other platforms besides Windows.

The audio guys, Andrew and Geoff, have done some really nice stuff in terms of sound. The audio aesthetic for the game is starting to come together. We have lots of rad footsteps and are doing some very interesting things in terms of environment sounds.

This is not a comprehensive account of everything everyone’s been doing; it is just what leaps to mind!

Every week we have a meeting with the architects where we plan the structures and landscapes on the island. Most of the week, the architects are generally off thinking about this stuff while we are building the game. During the weekly meeting we get together, talk about what concepts the architects have come up with, figure out how those might integrate with the game, make decisions about which ones to pursue, and iterate in detail on concepts we have already decided to pursue.

A while back we took some photos of this meeting, so I figured it would be fun to post them.

Here we are looking at a number of different possibilities for one of the areas in the game. I had already done the gameplay design, so we had a working location that was built out of blocky programmer stuff (if you see blocky things in the island update screenshots, it was something like that). So we knew the general shape of what we were trying to build and how it had to behave; the architects started with that and made proposals about what the location might be like in terms of its basic construction.

We chose one of these ideas and iterated on it a while. Other concerns, having to do with the overall layout of elements on the island, then caused us to change the size and shape of this location; once we had done this, the original design was felt to be a bit weak at the new size and shape, so we went back and heavily modified the concept. Only this past week have we converged on the final design and gotten a good first approximation into the game in a playable way. It’s pretty cool.

Because we’re working on a lot of things at once, we usually talk about a few different locations that are at different stages of development. At this particular meeting we were also looking at some early concepts of the visual design and modeling of the area where you start the game:

Last week, I got together with three other designers for a four-day intensive design retreat known as the Depth Jam. The other attendees were: Chris Hecker of Spy Party, Marc ten Bosch of Miegakure, and Daniel Benmergui of Storyteller.

This event was an experiment. Chris and I had been discussing the idea for a while, but it took us a couple of years to get around to it. One of my personal goals in setting up this event was to find a new way to stimulate my professional development, because the old ways were not cutting it any more.

The Depth Jam was designed in reaction to shortcomings of other game-related events. In order to explain the design choices behind the Depth Jam, I will speak critically of these other events, in order to highlight the problems that the Depth Jam is meant to address. If you are a fan of these events, organize some of them, or otherwise identify closely with them, then this will be uncomfortable. The best I can do here is to assure you that this isn’t attack-style negativity; it is criticism that comes from years of carefully considering these situations and thinking hard about how to make things better.

Conferences

When I first started working in video games I learned a lot from conferences and lectures. The few days I spent at the Computer Game Developers’ Conference in 1996 were eye-opening, even though I wasn’t comfortable enough as a game developer to know how to make effective use of that time. As years go by and we get better at what we do, a natural shift occurs: in the beginning, we are mostly deriving benefit from other attendees and presenters; later on, we are mostly providing benefit to other attendees, getting little out of it ourselves. I have been to the Game Developers Conference 17 times now. I find that I still do get something from attending, but it really takes a lot of work and there are very few people I can expect to learn from.

All the smart programmers I know complain about conferences and consider them basically useless (except for the smart programmers who are also on conference advisory boards). I certainly developed this kind of frustrated “there’s nothing good here to see” attitude in the early 2000s, but to mitigate this situation I shifted into being much more of a conference presenter than an attendee. A lot of creative energy went into planning new conference sessions and making them good. This helped extend the useful life of conferences, because I was learning a great deal by running sessions. After about eight years, though, this ran its course and I had gotten the bulk of what I was going to get from this arrangement.

Game Jams

In a typical game jam, developers gather for 2-4 days to do a working sprint, the goal being to produce a finished game entirely during the event. I was lucky enough to be around for the Indie Game Jam, which probably started the game jam trend (though I was often too busy working on last-minute GDC lectures to make jam games, sadly!)

From the first Indie Game Jam. Pictured: Brian Jacobson, Sean Barrett, Ken Demarest, Charles Bloom, Jonathan Blow, Brian Sharp, Doug Church.

The Indie Game Jams were very different from the jams we see today. The IGJ was founded on the idea of exploring the design ramifications of a crazy technical question; we would provide attendees with some code to accomplish some technical feat, then they would see where that would lead in terms of design. For example, the question for the first IGJ was “Graphics hardware is pretty fast now; what will people design if we give them an engine that can draw 100,000 little sprite guys on the screen at once?” We were picky about who we invited: attendees had to be designer-programmers who were actually good at programming, because dealing with new technology on a short timeframe can be very challenging.

Also from the first Indie Game Jam. Pictured: Art Min, Charles Bloom, Robin Walker, Thatcher Ulrich, Brian Jacobson, Zack Booth Simpson, and ... I don't know, Justin Hall? Charles Bloom again?

In contrast, contemporary game jams are more open. The idea is that it’s great to make a game, any game, and that even if you don’t manage to finish, you were still part of a community. This community spirit is often upheld as the best part of a game jam.

I think these jams can be really nice for beginners, to show people that making a game is something within reach, and to help them meet other people interested in making games. For experienced developers, though, I think these jams are not so good, because the jams are part of an overall social context that supports stagnation.

Experienced developers would do well to continually hone their craft and push beyond their comfort zone, but jams celebrate low expectations and provide Warm Community Feelings that create a comfort coccoon. “You made a game that is not very interesting? You didn’t finish a game at all? Well, you are still part of a community that likes you because you are participating in a game jam, and that is what is truly important.”

I don’t think there is anything inherently wrong with desiring a feeling of community, but we must take care to separate the notion of participating in the community from the notion of success. Participating in a game jam is not an indicator of success at game development — nor is attending the Independent Game Summit or Indiecade. I think that nice feelings are nice, but you want to build an ecosystem where the nice feelings coincide with behavior that will make developers robust and powerful and interesting in the long term. You don’t want nice feelings to act as a soporific. Activities that are good for beginners are often stagnant for intermediate developers.

What to do about this? I think if there were a category of game jams with higher expectations, so that advanced jams were differentiated from beginner jams, that would be a nice start. The IGJ model definitely asked more of participants, but I don’t think the IGJ is the right thing going into the future, because even if you are playing with challenging technology, the time-limited format prevents you from going deep. IGJ was nice in 2002, but today it would just contribute further to the problem outlined in Chris Hecker’s rant Please Finish Your Game. We see lots of wacky but shallow game designs all over the web, and it feels like a problem, or at least a vast sea of potential unreached.

As Chris says in his write-up about the Depth Jam, “game jams are shallow by design.” Because they are shallow, I don’t feel they are the right place for me to develop as a game design practitioner.

Retreat-Like Gatherings

There have been a few retreat-like gatherings for forward-thinking game designers: see for example Project Horseshoe and Phrontisterion. I have never been to these. I like the basic idea behind these kinds of events, but the execution seems to have endemic problems. I know people who have been to both of these events, but when I ask, they do not recommend attending the events. Certainly I have found Project Horseshoe’s written reports unhelpful.

The obvious problem with these events is that they are mostly just a bunch of talking. When people get together for a bunch of talking, most of them just say a bunch of bullshit (here I mean bullshit in the Harry G. Frankfurt sense) that is unfocused and untethered to reality. If it is not of critical importance whether what people say is right, then most of it is not going to be right.

I really like the retreat model as a basic template. Over the past few years I’ve been to a number of retreats, mostly for things like meditation or silent existential contemplation. There’s a lot good about going to a far-away place where you are not bothered by the concerns of everyday life. But looking at Phrontisterion and Horseshoe, the question arises: how does one design a retreat like this that is not just a bunch of talking?

I knew some of the answer, because I’ve seen success in dealing with a similar question in a neighboring realm:

Local Developer Meet-Ups

Many cities around the world have regular meet-ups; once a month or so, game developers get together at a bar or something, and just chat and be social. I generally don’t go to these because bars are antagonistic to interesting conversations and because attendees tend toward the neophyte side, which means we have the same problem as at conferences: there’s little benefit to be had for an experienced developer. But these events also have the Horseshoe problem, in that the conversation is not focused and doesn’t really matter, so people just say a bunch of nonsense. If all you want is to get drunk and be social, these events are fine, but they don’t do much for professional development.

Two years ago I started a series of monthly developer meetups in the San Francisco Bay Area; the idea was to keep discussion quality high by (a) holding the meetings in quiet places conducive to good discussion, like someone’s house (b) inviting only active game designers [this was not a "game industry" meeting, it was a "thoughtful game designer" meeting], and (c) starting each meeting with one of the attendees presenting their own work. After the presentation, we discuss that work specifically for a while; then at some point, the discussion naturally dissolves into separate, more-general discussions.

(I have been to some developer meet-ups that also began with presentations, but which to me did not manage to establish a culture of quality. One example was the Austin IGDA meetings back in 2002-2003. I think there were many subtle things preventing quality from rising, mostly having to do with intention of the event: the goal of the meeting was to drive attendance, not to be deeply interesting; also, they were “game industry” meetings, with all the associated issues. Also, scale matters: the Austin IGDA meetings were bigger, and they occurred in offices or at places like Dave & Buster’s, which did not encourage a personal connection to the presentation or the other attendees).

The meetings in the Bay Area were very successful at keeping discussion quality relatively high. The situation wasn’t perfect, but it was much better than your typical bar meeting. (I would have attended these regularly even if I were not involved in organizing them). Other attendees really enjoyed the meetings as well. After about a year, though, I stopped arranging the meetings because we seemed to have exhausted the supply of people willing to present, and I didn’t want to start having meetings that were not kicked off by solid presentations. Because everyone had a good time, there’s a reasonable probability that in the future we will pick these up again. I think we might be able to improve the discussions further by reducing the presenter-vs-audience asymmetry somehow (see the Depth Jam section below).

Key to the success of these events was having specific, concrete issues to talk about: a specific game being presented, so that discussion could be anchored to the details of that specific game. It’s also important that the game was being presented by its author; if the session is just someone talking about someone else’s game that they liked or didn’t like or just want to say something about, it is too easy for the discussion to be useless bullshit.

It seemed like this model could be applied to ground a retreat so that it would not just be a bunch of talking.

The Depth Jam idea

Now, we come to the actual Depth Jam. We settled on the basic idea very quickly: the jam would occur in a relaxing retreat-like environment. There would be a limited number of participants; four seemed like a good number. Each participant would have a good game that he has already been working on for a while, and which presents some deep and interesting problem he would like to solve; this problem serves as a focus for discussion. The fact that every particpant has a game under discussion means that every participant has “skin in the game”, which keeps discussion tethered and unfrivolous.

I’d like to emphasize this last point because it can be subtle but it is crucial. Suppose some people are showing their games and being criticized and generally having a rough time due to all the stress that happens naturally when having one’s creation dissected; and meanwhile, the people who are criticizing do not have any obligations, and they are just tossing in comments from the peanut gallery. This situation creates a weird imbalance. The comments and criticism will not be as thoughtful as they could be, yet they will be taken very hard by the people showing the games.

If everyone is having their creations dissected, there’s only one class of attendee instead of two. It is easier to empathize and avoid unnecessary harshness. People are going to be more careful that their ideas and criticisms are thoughtful, because they are acutely aware of wanting careful input when it comes to their own game.

The Format

Our first proposal for the Depth Jam had us spending one entire day on each game, so that we could dive into each game at maximum depth without context-switching. However, as the idea churned, we decided it would be beneficial to allow some iteration. Why not split the day into two time slots, so that each game gets half a day and we cycle through the four games twice? This would allow time to modify the games based on the discussion, which would give the discussion even more teeth: now we’re not just talking about a particular problem in a particular game (with some kind of tendency to wax philosophical), we’re actually figuring out how the author will address the problem here at the retreat, with the results of that approach to be plainly visible in the next session.

In our pre-jam planning meeting, we decided to go even further this way, breaking each day into four slots, cycling through all the games four times (so that each game had a two-hour slot each day). This seemed to work pretty well and it allowed a lot of iteration, but it might have been excessive. Daniel thinks there was too much context-switching and he might have been able to think more effectively if given more stability. My impression is that the talking was great for the first couple of days and degraded in quality toward the end (but was still worthwhile even then). At least two attendees did not have a problem with this, though, and found that the final discussions were very valuable for them.

For the next jam I would propose a mixed format: we’d start with two 4-slot days, just like we did this time, followed by a day with no-talking, all-working-and-quiet-and-taking-a-walk, followed by a final 4-slot day. (Or, perhaps I would lengthen the retreat to 5 days and put the rest day in the middle.)

The nice thing about the 4-slot-per-day format, which would not have been true of the 2-slot-per-day format, is that it gives us the maximum amount of calendar time for ideas to stew between iterations. I have long been appricative of the role of calendar time in good design. Sometimes it doesn’t matter how many hours you pack in trying to get things done; the good ideas will arrive unpredictably, and maybe you just need to allow time for this to happen. I was curious whether this principle would also be true on the timescale of a 4-day retreat. For me, at least, it was; I got my best idea in the shower on the morning of day 3, in response to discussions we’d had on day 2. During my session on day 3 we discussed and refined the idea, and on day 4 I showed an implementation of it.

Creature Comforts

We spent some money renting a nice beach house and ordering catered food; the cost for the event was around $5000. Chris goes into more detail about this in his write-up. You don’t need to spend any money on an event like this, but if you can afford it I recommend spending some money to help create a nice environment that minimizes stress and factors away concerns like “what are we going to eat for dinner” and “who is going to do the dishes”. The purpose is twofold: first, it helps you focus on the subject at hand; second, the minimization of external stresses helps you deal with the potential added stresses of working really hard and disagreeing with people all the time.

If you think that the Depth Jam will help you make more headway on even one deep problem than you would have otherwise, and thus make your game better, it’s easy to think that the game will also sell a little better and that costs in the neighborhood of $1250 per attendee are easily justified.

Attitude

Discussions can easily turn into arguments, or at least energy-sucking disagreements. As peoples’ energy gets drained, they become more irritable, so there’s a feedback loop lurking here. A little bit of this happened at our Depth Jam, but we saw it happening and course-corrected, so that the final day’s discussion was reasonable.

Prior to the jam, though, we had not thought much about this. I think it would be helpful in future for the attendees to go in knowing that irritability is likely to happen; the mere fact of this awareness probably helps the situation, and anyway, a little bit of psychologically-aware pacing at the beginning would have gone a long way.

Games and Attendees

It’s very important the attendees be capable both of participating in good discussions and acting on the discussions to improve their games within a short timeframe. This latter requirement basically limits attendees to being competent designer/programmers. If someone can’t program, it’s hard to see how he can participate meaningfully in this style of depth jam, because it would be very hard to iterate. Possibly if someone is a level/world/puzzle designer who can build scenarios quickly in UDK or Unity or whatever, it can be made to work, but I still think that person would be feeling the limitations of being unable to make algorithmic changes.

I don’t think that having teams of people would work, at least not for the format described here, because it would dilute the energy and bog down the iteration process. If you have one designer and one programmer trying to do the job that the other people are doing as single designer/programmers, your duo is probably going to have a hard time keeping up. If the jam is made up entirely of duos, it’s going to make discussions a lot messier and lower-energy (twice as many people talking about the same number of games). At least one of us felt that four people was already too many for high-quality discussions, because you keep having interesting ideas but have to wait for other people to stop talking in order to say them, and by that time the discussion may have moved on to a different topic.

We have tossed around some ideas about how to scale the jam to larger groups, but haven’t come up with anything that is fully convincing yet.

It’s important that the games be high-quality efforts that pose problems that everyone will be interested in. We were fortunate to have four very interesting games: Miegakure, a puzzle-platformer in four dimensions; Storyteller, a game about building stories through character interactions; Spy Party, a heavily player-skill-oriented game about subtlety and deception in human behavior; and The Witness, a first-person puzzle game with a heavy emphasis on nonverbal communication.

If time permits I may do a detailed write-up of the issues we discussed for each game and the resolutions we reached (though care must be taken here, as we don’t want to disclose aspects of these games that the designers would rather keep secret).

Conclusions

I am happy with the way this first Depth Jam went. I think we can certainly tweak the format to improve it, but already it is a useful tool in my further development as a game designer. I got much more out of this four-person, four-day event than I do from attending a conference. It seems appropriate to me to do a Depth Jam every six months. Provided we are organized enough to get the next one together, we’ll adjust the format and see how it goes!

See also:

Chris Hecker’s write-up
Daniel Benmergui’s write-up
Miegakure web site

In a strange twist of fate, the corner of the island that I chose long ago to be closest to the camera is the place that has taken longest to develop. As you can see, we are continually modifying this area but it’s still in a very rough state. Most other areas of the island are further along.

Lately I have been doing a little bit of playtesting and high-concept discussion (more on this later in the week). I’ve also been doing more work on the endgame. The modelers have been cranking away at making areas look better. I think we have another engine technology posting coming down the pipe.

In this instalment, Thiel takes the question of what kind of company to start and reframes it in terms of secrets.

Thiel’s point of view reminds me of the way I think about games in general. Quite often I go out and give speeches about game design or business, and I say things that some think are overly idealistic, or oblivious to the necessities of recipients’ situations, or just plain wrong. Certainly my advice usually runs counter to conventional wisdom. (Yet somehow, by following these principles, I seem to do okay.)

Thiel begins with the question: “What important truth do very few people agree with you on?” and goes on to define secrets, in this context, as “unpopular or unconventional truths”. If everyone knew these things and believed them, they wouldn’t be secrets.

I recommend this write-up to anyone who wants to think unconventionally about game design.

The changes are a little bit drastic in some spots, but overall not as drastic as I had expected. I had been worried about some issues in the natural flow of where players would naturally end up going, especially early in the game. It turns out we were able to solve these without changing too much. So between last island update and this one, for the most part, some areas were moved laterally or expanded, and some new elevation changes were introduced, but areas were not shuffled around; they are all more-or-less in the same place with relation to each other. So for the most part, things are converging, which is a good sign.

And, oh yeah, we are making that mountain a lot bigger. It is the messiest part of the island right now, and has been for a while. There are still a few things gameplay-wise to figure out here, then it will start to come together. My biggest questions in the area have to do with that white building near the bottom of the image; the puzzles inside that building will determine its overall structure, which will determine how it is worked into the surrounding terrain of the mountain. We have some proposals from the architects on what this building will look like, but it is waiting on me to finish the puzzle design.

Because we haven’t done it in a while, here’s the island from a different angle:

The current puzzle count for the whole game is 450.

Just before I left my last job I was working as a Lead/Art Director on a marketing driven project focused on selling as many units as possible. On one side I had high level decisions being made by stockholders whose focus is on turning a profit while on the other side there was the development team who truly believed they would be able to make a good game.

Also I was surrounded by schedules, performance reviews, hierarchical titles, producers, human resources, bonus system and all the other corporate systems that eventually dilute the organic process of doing something creative. While I understand the logic behind it I  realized that is not the reason why I joined the video games industry and that something wasn’t right.

Now, after these few months, the answers for these questions became clear. Its all based on trust. In this project, all the decisions we arrive at are very organic. Jonathan has a very clear vision of what he wants and the fact that we can see it and believe in it makes it extremely logical and simple.

Our passion for what we do (and what we would probably be doing on our free time if we weren’t being paid for it) gets distilled directly into the game in a very rewarding process.How much and how we give it depends entirely on us and that creates a natural challenge and pressure that most AAA companies cannot provide.

This trust leads to respect and it creates a very unique work environment. This is something that I’ve never seen with all the project management, task scheduling and scrum meetings that try to fix an inherently broken game. I don’t think video games can be produced like an assembly line, at least not games that have something meaningful to say, and I’m glad that I found people that are doing it the right way.

All this may sound trivial but probably 90% of the game developers that I know will understand how big of an issue this is. I still believe big companies can change their ways and make the same profit if not more, but it has to start from the bottom, the people who are hard at work making the games.

As for the second part, I would like to share what we have been trying to do with the many island panels. As you might already know the player has to solve different puzzles using these panel interfaces scattered around the island.

I started my research by looking at what materials they could be made of, what technology they would use and how they would be assembled and how all this could relate with the different areas and narrative of the game.

This first set of images shows some proposals of how the panel screens could possibly look. I tried not to focus so much on the gameplay but more on ideas that could lead to further discussions among the team.

After some solid feedback and a couple of art meetings I was able to understand what was working or not and decided to do a new set of proposals. Also this time I tried to focus on the more technical aspect of getting these panels to work in the game and if the effort we would have to put into them would be worthwhile.

The same amount of work went into the structure of the panels and how they would be assembled. This time, working with the architects gave valuable input into understanding how they would actually be built in the environment and the amount of work required. On this image you can see a screen capture of several prototypes before being tested ingame.

These are usually modelled very fast with just base colors so I can do several iterations until I find something that I’m happy with.
And finally here is how some of them are looking in-game. They are still in concept but by placing them in the correct context makes it a lot easier to understand what is working or not.

But today we are going to have the first in a series of meetings wherein we think pretty hard about the topography of the island and what the flow of natural curious investigation will tend to be from various spots. Expect the island to look pretty different in the next update! (Or maybe it will turn out that things are mostly fine…)

Lately, I have been working on some of the more sophisticated puzzles in the game. These are things that were at least sort-of done already, but I wanted to revise the designs to improve them — to ensure that the puzzles are as interesting as possible and built as closely as possible around the core themes of the areas that contain them. I am happy with the way these are coming along so far, but there is definitely still some work to do here.

The art team has been working further to nail down the style of the game, now focusing on cliffs and plants and other natural things, as well as control panels and other bits of machinery. We’re also taking a few buildings that the architects have finished designing, and starting to build out the geometry as one really would for a video game. (The architects give us buildings modeled in SketchUp, and they tend to come into the game a bit messy, but also, not modeled the way an experienced video game modeler would do it).

On the programming side, we’ve got lots done lately. Andy has added a software occlusion culling system, which should help with frame rates in the game. He’s also been playing with culling for water reflections. Salvador got animations working in the exporter and in the game, so now we can play animations on entities, which is very useful. (We worked on the game for years and built something fully playable without this ability!) I have been going back and doing the gameplay work for animated entities, as some things have to change in the gameplay logic. In all it is a very nice improvement. Salvador is now working on asset streaming, which will help the game start up faster and run better (and help it run at all on lower-end platforms like the iPad). Ignacio’s been doing a whole lot of fixes to various systems and hammering away at the todo list. One of these things has been better control for the tone mapping process. We now get a graph of tone mapping curve:

And we also get the ability to export screenshots into Photoshop that have a color key on them; then the colors can be tweaked in Photoshop and re-imported into the game to alter tone mapper settings. This is in many cases more convenient than playing with those sliders.

We had to do a lot of work to get the game back into playable condition; a lot of things had been broken since the press tour last August / September. (When you are making sweeping changes to a big and complicated game world, it is pretty easy for things to get bent out of shape). So the GDC served as a good internal deadline to have the game banged back into shape.

I didn’t expect a lot of press coverage to come out of this, because at the GDC there’s just a lot of stuff going on, a lot of things for people to pay attention to and write about. So I had been doing this mostly as an opportunity to keep the press up-to-date with the game, see how it is evolving graphically, etc. But, to my surprise, we got a lot of coverage. Here are links to what people are saying about the game lately:

First, a video interview on Gamespot (excerpted; we probably talked for 45 minutes or an hour).

Kirk Hamilton at Kotaku finds a musical treatment for The Witness’ game design philosophy.

Ben Kuchera at the Penny Arcade Report, with a piece about online distribution channels, and another biz-focused one.

Ben Gilbert at Joystiq, with a design-oriented posting and also a more biz-like one.

Alex Rubens from G4 with an overall preview of the game.

Daniel Starkey of Destructoid, with a preview as well.

As you can see from the write-ups, people seemed to really dig the game. So that is all good.

Keep in mind that, because I am known for my previous game and and the reporters are there talking directly to me, the tone that tends to get adopted in these write-ups makes it sound as though I were making the game myself. That’s not the case; there’s a good-sized team of very talented people building the game! (This reminds me that we need to update the About page to list everyone who is currently working on the project.)

When textures are static a common solution is to pre-process them in an attempt to eliminate the edge seams. In a short siggraph sketch, John Isidoro proposed averaging cube map edge texels across edges and obscuring the effect of the averaging by adjusting the intensity of the nearby texels using various methods. These methods are implemented in AMD’s CubeMapGen, whose source code is now publicly available online. While this seems like a good idea, a few minutes experimenting with CubeMapGen make it obvious that it does not always work very well!

Embedded Texture Borders

A very simple solution that even works for dynamic cube maps is to slightly increase the FOV of the perspective projection so that the edges of adjacent faces match up exactly. Ysaneya shows that in order to achieve that, the FOV needs to be tweaked as follows:

fov = 2.0 * atan(n / (n - 0.5))

where n is the resolution of the cube map.

What this is essentially doing is to scale down the face images by one texel and padding them with a border of texels that is shared between adjacent faces. Since the texels at the face edges are now identical the seams are gone.

In practice this is much trickier than it sounds. While the fragments at the adjacent face borders should sample the scene in the same direction, rasterization rules do not guarantee that in both cases the rasterized fragments will match.

However, if we take this idea to the realm of offline cube map generation, we can easily guarantee exact results. Cube maps are often used to store directional functions. Each texel has an associated uv coordinate within the cube map face, from which we derive a direction vector that is then used to sample our directional function. Examples of such functions include expensive BRDFs that we would like to precompute, or an environment map sampled using angular extent filtering.

Usually these uv coordinates are computed so that the resulting direction vectors point to the texel centers. For an integer texel coordinate x in the [0,n-1] range we map it to a floating point coordinate u in the [-1, 1] range as follows:

map_1(x) = (x + 0.5) * 2 / n - 1

We then obtain the corresponding direction vector as follows:

dir = normalize(faceVector + faceU * map_1(x) + faceV * map_1(y)

When doing that, the texels at the borders do not map to -1 and 1 exactly, but to:

map(0) = -1 + 1/n
map(n-1) = 1 - 1/n

In our case we want the edges of each face to match up exactly to they result in the same direction vectors. That can be achieved with a function like this:

map_2(x) = 2 * x / (n - 1) - 1

If we use this map to sample our directional function, the resulting cube map is seamless, but the face images are scaled down uniformly. In the first case the slope of the map is:

map_1'(x) = 2 / n

but in the second case it is slightly different:

map_2'(x) = 2 / (n - 1)

This technique works very well at high resolutions. When n is sufficiently high, the change in slope between map_1 and map_2 becomes minimal. However, at low resolutions the stretching on the interior of the face can become noticeable.

A better solution is to stretch the image only in the proximity of the edges. That can be achieved warping the uv face coordinates with a cubic polynomial of this form:

warp3(x) = ax^3 + x

We can compose this function with our original mapping. The result around the origin is close to a linear identity, but we can adjust a to stretch the function closer to the face edges. In our case we want the values at 1-1/n to produce 1 instead, so we can easily determine the value of a by solving:

warp3(1-1/n) = ax^3 + x = 1

which gives us:

a = n^2 / (n-1)^3

I implemented the linear stretch and cubic warping methods in NVTT and they often produce better results than the methods available in AMD’s CubeMapGen. However, I was not entirely satisfied. While this removed the zero-order discontinuity, it introduced a first-order discontinuity that in some cases was even more noticeable than the artifacts it was intended to remove.

You can hover the cursor over the following image to show how the warp edge fixup method eliminates the discontinuities, but sometimes still results in visible artifacts:

Any edge fixup method is going to force the slope of the color gradient across the edge to be zero, because it needs to duplicate the border texels. The eye seems to be very sensible to this form of discontinuity and it’s questionable whether this is better than the original artifact. Maybe other warp functions would make the discontinuity less obvious, or maybe it could be smoothed like Isidoro’s method do. At the time I implemented this I thought the remaining artifacts did not deserve more attention and moved on to other tasks.

Modifed Texture Lookup

However, a few days ago Sebastien Lagarde integrated these methods in AMD’s CubeMapGen. See this post for more results and comparisons against other methods. That got me thinking again about this and then I realized that the only thing that needs to be done to avoid the seams is to modify the texture coordinates at runtime the same way we modify them during the offline cube map evaluation. At first I thought that would be impractical, because it would require projecting the texture coordinates onto the cube map faces, but turns out that the resulting math is very simple. In the case of the uniform stretch that I first suggested, the transform required at runtime is just a conditional per-component multiplication:

float3 fix_cube_lookup(float3 v) {
   float M = max(max(abs(v.x), abs(v.y)), abs(v.z));
   float scale = (cube_size - 1) / cube_size;
   if (abs(v.x) != M) v.x *= scale;
   if (abs(v.y) != M) v.y *= scale;
   if (abs(v.z) != M) v.z *= scale;
   return v;
}

One problem is that we need to know the size of the cube map face in advance, but every mipmap has a different size and we may not know what mipmap is going to be sampled in advance. So, this method only works when explicit LOD is used.

Another issue is that with trilinear filtering enabled, the hardware samples from two contiguous mipmap levels. Ideally we would have to use a different scale factor for each mipmap level. That could be achieved sampling them separately and combining the result manually, but in practice, using the same scale for both levels seems to produce fairly good results. We can easily find a scale factor that works well for fractional LODs as a function of the LOD value and the size of the top level mipmap:

float scale = 1 - exp2(lod) / cube_size;
if (abs(v.x) != M) v.x *= scale;
if (abs(v.y) != M) v.y *= scale;
if (abs(v.z) != M) v.z *= scale;

If you are using cube maps to store prefiltered environment maps, chances are you are computing the cube map LOD from the specular power using log2(specular_power). If that’s the case, the two transcendental instructions cancel out and the scale becomes a linear function of the specular power.

The images below show the results using the warp filtering method (these were chosen to highlight the artifacts of the warp method). Hover the cursor over the images to visualize the results of the new approach:

I’d like to thank Sebastien Lagarde for his valuable feedback while testing these ideas and for providing the nice images accompanying this article.

Note: This article is also published at AltDevBlogADay.

The current puzzle count is 440.

As expected in the previous update, over the past week or so I went back and worked on the endgame again. I built an original version of the endgame last summer; I had one friend playtest it, and it worked out pretty well, though I felt it wasn’t quite epic enough. I didn’t know what more to do with it at the time, so I just let it sit, and went to thinking about other parts of the game.

Lately I had more ideas for the endgame, so I went into personal crunch mode and rebuilt into something more epic. This was one of the most productive periods of my working life (maybe the most productive) and I think this area, all taken together, may be the most interesting thing I have ever designed. However, it may be a little too difficult for the player, at the moment! Today I playtested this endgame and one other area of the game, with a game designer friend of mine, and it took nine and a half hours to play through these two areas (6 hours for the endgame, 3.5 hours for the other area). I think this is a little long, so I am going to be tuning these areas and cleaning them up a little bit. This weekend hopefully I can get in some playtest time with Jeff and Casey of Jeff and Casey Time, so if that happens we will see how it goes.

(It’s interesting; I haven’t done a playtest of The Witness for a while, but I was still thinking of it as a 10-15 hour game, and that’s what I tell people when they ask. But if these two areas by themselves, even after being cleaned up, are 6-7 hours of playtime, that means the whole game is … … much longer. Sometime in a month or two, I’ll do some playtests with people who have never seen the game before, and we’ll see how long it really is.)

Meanwhile, on the art side, we’ve been working on the style of the game; expect a future blog post as we come to conclusions on that.

Tech-wise, Andy has been working on performance stuff. First he made raycasting much faster; til now the game had been relatively lazy about spatial organization. Andy implemented a quadtree for finding entities in the world and enabled k-d tree generation and serialization for casting rays against individual meshes. (We had k-d tree code already as part of a library but weren’t using it.) Salvador is working on in-game animation playback, as well as exporting; until now we have been making do without animations, but once this is in the game it will help us with a number of issues.

Our gift to you, on this very special Valentine’s Day, is the worldwide release of the long-awaited game Dear Esther.

If you haven’t heard of Dear Esther, watch this:

Click here to view the video on YouTube.

(Or hey, watch the trailer even if you are quite familiar with the game; the trailer is beautiful and worthy of multiple viewings.)

We expect public reception of this game to run wide: some will love it, and others will be very concerned about whether this thing can be called a game and what that means. So far, this has certainly been the case in pre-release reviews.

Game Informer scored the game an 8/10, saying: “You should consider checking out Dear Esther the same way you’d appraise a film. If you’re interested in absorbing an intellectual story and gorgeous visuals without having to exert a drop of effort, take a chance on this curious experiment.”

VideoGamer.com scored the game a 9: “Discovery is such an important part of Dear Esther, especially when everything is so phenomenally pretty.”

Meanwhile, Destructoid gave the game a lowly 4.5/10: “It’s as if it wants to be a part of this wonderful medium of ours without asking itself why, which is exactly why you should seek it out and learn from its failures as a game enthusiast, critic, or developer.”

We like that there’s such a big difference of opinion because it means the game is breaking new ground. It’s playing in territory that is not safe; there is no established understanding there.

Dear Esther is a game that no publisher would have funded. Dan, Rob, Jessica and the other associates of thechineseroom have done an excellent job putting together a beautiful game. We are happy to be backing it; we hope you enjoy playing it.

If you’d like more information about Dear Esther, here’s an interview with Dan, and here’s a link the game’s page on Steam.

Update: Dear Esther has been wildly successful, selling 16,000 copies in under 24 hours. As an investment, it reached profitability in 5.5 hours. More details are available at a new post on the Indie Fund blog.

This is because this corner of the world is one of the last areas that we haven’t really figured out, in terms of what goes where. There is sort of just some random stuff dumped there right now. That will change pretty soon.

Expect the next few island snapshots to show a lot of big changes. We’ve reached the point where the basic gameplay is settled, I’m refining many of the specific puzzles, and we’ve got at least rough drafts of many of the architectural layouts. That means we can start planning the island a bit better. Over the next month or so, we’re going to be changing the shape of the island to something a little more geologically plausible, something that carries more information. It’s going to be interesting to do this without disrupting the gameplay layout of the island too much.

The current puzzle count is 427. In January I scrapped and re-designed an area and its 60-or-so puzzles. The previous area was inside dark caves, which is a bit cliche, and the gameplay hook that motivated my setting that area in caves, which I had hoped would be cool, didn’t work very well. In addition, I felt that the puzzles themselves, while fitting into the rest of the game, were the weakest part of the game. I’d been feeling this way for a long time, but I didn’t have a good solution for it til late December. I moved the location and redesigned the puzzles. Now the theme of the location is totally different, has a gameplay hook that works way better than the cave thing, and is very cool. The puzzles situated in this area are also much stronger than before, because I found a way to change the rules of those puzzles to make them much tighter. They’d felt wishy-washy before (people would play through that area and, after finishing, lack clarity on what the puzzles were saying. That’s no good.)

We certainly could have shipped the game with the old area in the caves with the old puzzles, and probably shipped the game at least a month sooner, but an important skill in building a beautiful game is knowing when something just isn’t good enough.

Next I am going back to the endgame, which I did my first pass at last summer sometime; I’ve got a gameplay hook to add which I think will bring it over the top and make it extra-good. I’ll be working on that over the next few weeks while also working with the landscape architects on the island revision.

Also, we’ve got a new programmer starting soon (this week or next!) Combined with our other new addition, who’s been here a few weeks (Andrew Smith, ex-Zipper and -Oddworld; hi Andy!) we will have quite a bit of tech manpower to bring to bear on finishing this game.

The trailer has just been released:

Click here to view the video on YouTube.

See more about Dear Esther at the official site.

This will be the second Indie Fund game to release. Just a few days ago we announced that Q.U.B.E., the first funded game, is already a financial success. It appears obvious that Dear Esther will be as well.

A small team with limited manpower, we wanted a low-effort way to use our existing version control system (svn) to facilitate concurrent world editing for our open-world 3D game (i.e. we desire that multiple users can edit the world at the same time and merge their changes).  We developed a system wherein the data for each entity is stored as an individual text file.  This sounds crazy, but it works well: the use of individual files gives us robustness in the face of merge conflicts, and despite the large number of files, the game starts up quickly.  Entities are identified using integer serial numbers.  We have a scheme for allocating ranges of these numbers to individual users so there is no overlap (though this scheme could certainly be better).  The system seems now to be robust enough to last us through shipping the game, with seven people and one automated build process concurrently editing the world across 18 different computers.

We don’t know whether this method would work for AAA-sized teams, though it seems that it might with some straightforward extensions.  But for us the method works surprisingly well, given how low-tech it is.

About the Game

We have been developing a 3D open-world game called The Witness since late 2008.  For much of that time, development happened in a preproduction kind of way, where not many people were working on the game.  Recently, though, we went into more of a shipping mode, so the number of developers has grown: we’re up to about 10 or 11 people now (pretty big for a non-high-budget, independent game).  Not all of these people directly touch the game data, but about 7 do now, and this number will rise to 9 in the near future.

The design of the game requires the world to be spatially compact, relative to other open-world games, so it is far from Skyrim-esque in terms of area or number of entities.  At the time of this writing, there are 10,422 entities in the game world.  I expect this number to grow before the game is done, but probably not by more than 2x or 3x the current amount.

Because the world is not too big, and some game-level events require entities to coordinate across the length of the world, it makes sense to keep all the entities in one space, that is to say, there is no zoning of the world or partitioning of it into separate sets of data.  This makes the engine simpler, and it reduces the number of implementation details that we need to think about when editing the world.  It also introduces a bit of a challenge, because it requires us to support concurrent editing of the world without an explicit control structure governing who can edit what.  One such control structure might be: if you imagine the world were divided into a 10×10 grid; so there are 100 different zones; and where anyone editing the world must lock whatever zones they want to edit, make their changes, check them in, then unlock those zones; this would provide a very clear method of ordering world edits.  (Because of the locking mechanism, two people would not be able to change the same world entity at the same time, so there doesn’t have to be any resolution process to decide whose changes are authoritative). However, the straightforward implementation of such a control scheme would make for inconvenient workflow, as it is tedious to lock and unlock things.  Even with further implementation work and a well-streamlined UI, there would still be big inconveniences (I really want to finish building a particular puzzle right now, but a corner of that puzzle just borders on some region that a different user has locked, and he is sick this week so he probably won’t finish his work and unlock it until next week.  Or, if my network connection is flaky then I may not be able to edit; if I am off the net then I definitely could not edit.)

Thus, rather than implementing some kind of manual locking, I found it preferable to envision a system that Just Magically Works.  Since we already used the Subversion source-control system to manage source code and binary source assets such as texture maps and meshes, it didn’t seem like too big of a stretch for Subversion to handle entity data as well.

The World Editor and Entity Data

Our world editor is built into the game; you press a key to open it at any time.  Here’s what it looks like:

The editor view is rendered in exactly the same way as the gameplay view, but you fly the camera around and select entities (I have selected the red metal door at the center of the scene, so it is surrounded by a violet wireframe).  In the upper right, I have a panel where I can edit all the relevant properties of this Door; the properties are organized into four tabs and here you can see only one, the “Common” properties.

These properties, which you can edit by typing into the fields of this Entity panel, are all that the game needs to instantiate and render this particular entity (apart from heavier-weight shared assets like meshes and texture maps).  If you serialize all these fields, then you can unserialize them and generate a duplicate of this Door.  That’s how the editor saves the world state: serialize every entity, and for each entity, serialize all fields.

Originally, the game saved all entities into one file in a binary format.  This is most efficient, CPU-performance-wise, and the natural thing that most performance-minded programmers would do first.  Unfortunately, though, svn can’t make sense of such a file, so if two people make edits, even to disjoint parts of the world, svn is likely to produce a conflict; once that happens, it’s basically impossible to resolve the situation without throwing away someone’s changes or taking drastic data recovery steps.

Our natural first step was to change this file from binary into a textual format.  Originally I had worries about the load-time performance of parsing a textual file, but this turned out not to be an issue.  I decided to use a custom file format, because the demands in this case are very simple, and we have maximum control over the format and can modify it to suit our tastes.

Many modern programmers seem to have some kind of knee-jerk inclination to use XML whenever a textual format is desired, but I think XML is one of the worst file formats ever created, and I have no idea why anyone uses it for anything at all, except that perhaps they drank the XML Kool-Aid or have XML Stockholm Syndrome.  Yes, there are XML-specific tools out there in the world, and whatever, but I didn’t see how any such tool would be of practical use to us.  One of our primary concerns is human-readability, so that people can make intelligent decisions when attempting to resolve revision control conflicts.  XML is not particularly readable, but we can modify our own format to be as readable as it needs to be in response to whatever real-world problems arise.  You’ll see a snippet below of the format we use.

Evolution of the Textual Format

The first version of our textual format was a straightforward adaptation of our binary format.  The gameplay-level code that saves and loads entities is the same for either format; the only difference is just a boolean parameter, choosing textual or binary, which is passed to the lower-level buffering code.  After the buffer is done being packed, it’s written to a file.

In our engine, entities are generally referenced using integer serial numbers (that act as handles).  These numbers are the same when serialized as they are at runtime.  Originally, in the binary file, the order in which the entities were written didn’t matter.  But if we want to handle concurrent edits, we want to make merging as easy as possible for the source control system, so we sort entities by serial number, writing them into the file in increasing order.  Thus we didn’t generate any spurious file changes due to haphazardly-changing serialization order.

In theory, Subversion could now handle simultaneous world edits and just merge them, the same way it does with our source code.  Unsurprisingly, though, this initial attempt was not sufficient.  Though the file was now textual, it contained a bare minimum of annotations and concessions to readability.  Though I knew I would eventually want to add more annotations, I didn’t want to go overboard if it wasn’t necessary, because these inflate the file size and the time required to parse the files (though I never measured by how much, so it’s likely the increases are negligible performance-wise and I was engaging in classic premature optimization).  As it happens, the kind of redundant information that increases human-readability also helps the source control system understand file changes unambiguously, so as we made the format more-readable, the number of conflicts fell.

Here’s what our file format currently looks like, for entity #6555 (the Door we had selected in the editor screenshot).

Door
72
6555
; position
    46.103531 : 42386a04
    108.079681 : 42d828cc
    8.694907 : 410b1e57
; orientation
    0.000000 : 0
    0.000000 : 0
    -0.069125 : bd8d9184
    0.997608 : 3f7f633d
; entity_flags
    160
; entity_name
    !
; group_id
    4059
; mount_parent_id
    6554
; mount_position
    0.035274 : 3d107ba5
    0.006069 : 3bc6e19c
    0.000038 : 381ffffc
; mount_orientation
    0.000000 : 0
    0.000000 : 0
    0.000027 : 37e5f004
    1.000000 : 3f800000

(The first line, “Door”, is the type name; 72 is the entity data version, which is used to load older versions of entities as their properties change over time; 6555 is the entity’s ID.  root_z and cluster_id (seen in the screenshot) are not saved into the file, because these properties have metadata that say they are only saved or loaded under certain conditions which are not true here.  For the sake of making the listing short, I’ve shown only the properties listed on that first tab in the screenshot.)
You’ll note that the floating-point numbers are written out a bit oddly, as in this first line under ‘position’:

    46.103531 : 42386a04

This is how we preserve the exact values of floating-point numbers despite saving and loading them in a text file.  If you do not take some kind of precaution, and use naive string formatting and parsing, your floating-point values are likely to change by an LSB or two, which can be troublesome.

The number on the left is a straightforward decimal representation of the value, written with a reasonable number of digits.  The : tells us that we have another representation to look at; the number on the right, which the : introduces, is the hexadecimal representation of the same number.  The IEEE 754 floating-point standard defines this representation very specifically, so we can use this hexadecimal representation to reproduce the number exactly, even across different hardware platforms.  The decimal representation is mostly just here to aid readability.  However, if we decide we want to hand-edit the file, we can just delete the : and everything after it, changing that first number to a 47.5 or something; seeing no :, the game will parse the decimal representation and use that instead.

There are other ways to solve this floating-point exactness problem; for example, you can implement an algorithm that reads and writes the decimal representation in a manner that carefully preserves the values.  This is complicated, but if you’re interested in that kind of thing, here’s what Chris Hecker says:

Everybody just uses David Gay’s code:
http://www.netlib.org/fp/dtoa.c
http://www.netlib.org/fp/g_fmt.c

You need to get the flags set right on the compiler for it to work. /fp:precise at least, I also do #pragma optimize (“p”,on), and make sure you write a test program for it with your compiler settings.

(Twitchy, and look at how much code that is!  What we did is a couple of lines.) In a document on his web site, Charles Bloom discusses a scheme that was in use at Oddworld Inhabitants, similar to what we’re doing in The Witness but involving a comparison between the two representations:

… Plain text extraction of floats drifts because of the ASCII conversions; to prevent this, we simply store the binary version in the text file in addition to the human readable version; if they are reasonably close to each other, we use the binary version instead so that floats can be persisted exactly.

Subversion Confusion

After switching to this format, conflicts still happened.  I categorize conflicts into two types: “legitimate”, where two people edit the same entity in contradictory ways and the source control system doesn’t know who to believe; and “illegitimate”, where Subversion was just getting confused.  We did have a number of legitimate conflicts for a while, yet which were not the users’ fault, because our world editor tended to do things that are bad in a concurrent-editing environment.  (See the Logistics section at the end of this article for some examples).  But even after we ironed these out, we still found that Subversion was still getting confused.

Here’s an example of a common problem: suppose we start with a world containing data on entities A, B, and D, so our file contains: ABD.  You edit the file and delete entity B.  I edit the file and also delete entity B, but add another entity C.  (That we both deleted B may be a very common use-case, especially if this is due to some automatic editor function).  So, you want to check in a file that just says: ABD – B = AD, so the new result is AD.  I want to check in a file that says: ABD – B + C = ACD (the C goes in the middle, because remember, the entities are sorted!)  In theory this is an easy merge: the correct result is just ACD.  But Subversion would get confused in cases like this, requiring the user to hand-merge the file.

This is a disaster, because even with a visual merge tool such as the one that comes with TortoiseSVN, you don’t want non-engine-programmers trying to merge your entity data file.  Even if that file is extremely readable, at some point someone will make a mistake; the file will fail to parse, and then you have a data recovery problem that requires the attention of an engine programmer.  When this happens it kills the productivity of two people (the engine guy, because he has to merge the data, and also the guy who generated the conflict, because until it gets resolved his game is in an un-runnable state, and anyway, the engine guy has probably commandeered the other guy’s computer to fix this).

After we had this problem a few times too many, Ignacio suggested moving to a more-structured file format like JSON, which, in theory, source-control systems would handle more solidly.  But I thought that this would only reduce the frequency of the problem, but not solve anything fundamental; even if we achieved a situation of only generating legitimate conflicts, each conflict would still be a potential game-crippling disaster.

Splitting the File

It occurred to me, finally, that it could hardly get cleaner than to store each entity in its own file.  This initial thought might cause a performance-oriented programmer to recoil in horror; we have 10,422 entities right now, and it must be very slow to parse and load 10,000 text files every time the game starts up!

But I knew that we already had over 4,000 individual files laying around for assets like meshes and texture maps, currently loaded individually, the idea being that at ship time we would combine these into one package so as to avoid extra disk seeks and the like.  But on our development machines, these files still loaded quickly.  I figured that if we can already load 4,000 big files, why not try loading an extra 10,000 small files?  If loading all these files turned out to be slow, we could resort to this only when entity data has changed via a source control update; after loading all the text files, we can save out a single binary file containing all the entity data, which would be much faster to load, just like we used to do before we supported concurrent editing.

I tried this and it worked great.  So, now we have a data folder called ‘entities’, which holds one file per serialized entity.  The Door selected in the editor screenshot is stored in a file named after its entity ID, “6555.entity_text”, which contains the text you see in the listing above.

As mentioned, there are currently 10422 entity files; the text files take up about 8MB of space, whereas the binary file is 1.4MB.  We’re not making any effort to compress either of these formats; they are very small compared to the rest of our data.

Benefits of Using Individual Files

Storing each entity in a separate file solved our problems with merging: both the illegitimate source control confusions as well as the problems due to hand-guided file merging.

The illegitimate conflicts go away because transactions become very clear.  If I delete an entity, and you delete an entity, then we both just issue a file delete command, and it’s very easy for the source control system to see that we agree on the action.  Furthermore, if we both delete B but I add C, because the change to C is happening in a completely separate file, there is no way for the system to get confused about what happened.

Legitimate conflicts do still happen, as multiple people edit the same entities, but these cases are much easier to handle than they were before.  The usual response is to either revert one whole entity or stomp it with your own.  Because we usually want to treat individual entities atomically — using either one person’s version of that entity or the other’s — individual files are a great match, because now the usual actions we want to perform are top-level source control commands (usually from the TortoiseSVN pulldown menu); before, this wasn’t true, and merging a single entity in an atomic way required careful use of the merge tool.  If an entity ever gets corrupted somehow, for whatever reason, we can just revert that one file, knowing the damage is very limited in scope; before we’ve reverted it, the worst that can happen is that that single entity will not load; it won’t prevent the rest of the entities in the world from loading.

It would be difficult to overemphasize the robustness gained.  I feel that these sentences do not quite convey the subtle magic; it feels a little like the state change when a material transitions from a liquid to a solid.

Sometimes, someone will update their game data in a non-vigilant way; not noticing that a conflict happened, so they won’t resolve it; and then they will run the game.  Before, this would have been a disaster; Subversion inserts annotations into text files to try to help humans manually resolve conflicts (this is an unfortunate throwback to programs like SCCS), which would have caused much of the world to fail to parse.  Now, though, the damage from these annotations is constrained to individual entities. It’s also easier to detect: if there is a conflict in the file for our Door #6555, then in addition to writing these annotations into 6555.entity_text (making it unparseable by the game), Subversion will also write out files 6555.entity_text.mine and 6555.entity_text.theirs, which contain the two intact but conflicting versions of the file.  The game can detect these very straightforwardly, put an error message on the HUD, and even choose one to load by default.  It is all very clean.

This is one of those interesting cases where reality-in-practice is starkly different from computer-science-in-theory.  In theory, a directory is just a data structure organizing a bunch of black-box data, and we happen to view this organization in a way that looks like a list of files; we could just do the same thing interior to our own file format, and it would be no big deal.  In practice, due to the way our tools work, these two arrangements are tremendously different.  The individual files just feel solid and reliable to work with, whereas one big structured file is always going to be precarious.

Startup Procedure

When the game starts up, we need to load all the entities from disk.  In the simplest case, this is straightforward: we just iterate through every file in the entities directory and load it.  However, as I mentioned, we wanted to keep a binary version of the entities that is preferred by the game if it exists and is up to date; partly, this might help startup speed a little (we don’t have to parse all these strings), but mainly, that is the file format and code path we are going to use when the game ships, and we don’t want that code to rot in the meantime.  Using it every day is the best way to prevent it from rotting.

So: when the game first starts up, we check for the binary entity package.  If it’s not there, we load all the individual entities files, then immediately save out the binary package.  (This binary package is a private, local file; if we were to add it to source control, everyone would always conflict with everyone else.)  But suppose we start up and the binary package is already there (which it usually will be).  We need then to determine if any of the textual entities have changed since the binary package was built.  The most-general way to do this is just by looking at the file timestamps; if we were to rely on some other signal being written to the disk that says “hey, entity data has changed,” then we would only be allowed to modify the entity files using sanctioned and customized tools; that would be annoying and would cause all kinds of robustness issues.

In concrete terms: we want to know the timestamp of the newest .entity_text file, inclusive of the directory containing these files (if we just did a Subversion update that only deleted entities, then all the .entity_text timestamps will be old, but the directory timestamp will be new; we want to ensure we properly rebuild the binary file in this case!)  To find the newest timestamp, we just iterate through all the timestamps in a brute-force way:

bool os_get_newest_file_modification_time(char *input_dir_name, Gamelib_Int64 *file_time_return) {
    //
    // We include the modification time of the directory as well.
    //

    char *dir_name = mprintf("%s/*", input_dir_name);
    Free_On_Return(dir_name);
    if (strlen(dir_name) >= MY_MAX_PATH - 5) return false;

    //
    // best_time starts as the time for the directory (if it exists!)
    //
    Gamelib_Int64 best_time = 0;
    bool success = get_file_modification_time(input_dir_name, &best_time);
    if (!success) {
        Log::print("Unable to get modification time for directory '%s'!\n", input_dir_name);
    }

    //
    // Now consider the file timestamps.
    //
    WIN32_FIND_DATA find_data;
    HANDLE handle = FindFirstFile(dir_name, &find_data);
    if (handle == INVALID_HANDLE_VALUE) return false;

    while (1) {
        if (find_data.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
        } else {
            Gamelib_Int64 file_time = to64(find_data.ftLastWriteTime);
            if (file_time > best_time) best_time = file_time;
        }

        BOOL success = FindNextFile(handle, &find_data);
        if (!success) break;
    }

    FindClose(handle);

    if (file_time_return) *file_time_return = best_time;

    return true;
}

bool get_file_modification_time(const char *name, Gamelib_Int64 *result_return) {
    HANDLE h = CreateFile(name, 0, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
                          NULL, OPEN_EXISTING, FILE_ATTRIBUTE_READONLY | FILE_FLAG_BACKUP_SEMANTICS,
                          NULL);
    if (h == INVALID_HANDLE_VALUE) return false;

    FILETIME write_time;
    GetFileTime(h, NULL, NULL, &write_time);
    CloseHandle(h);

    if (result_return) *result_return = to64(write_time);
    return true;
}

You’ll notice that this is Windows-specific code, but we don’t have to worry about porting it, because we only run the world editor on Windows.  Fortunately, in Windows, the file timestamp is right there in the directory data, so we don’t have to open each file.  (At least in NTFS this is true, rather than being an API illusion; the timestamps are stored in the Master File Table [MFT].)

What’s the performance like?  I typically run the game on two different machines: a high-performance Tower PC (CPU: Intel Core i7-970, hard drive: OCZ Vertex 3 SSD, RAM: 12GB) and a gaming Laptop (CPU: Intel Core i7-2820QM, hard drive: Intel SSD 500 [SSDSC2MH250A2], RAM: 12GB).

The time spent in os_get_newest_file_modification_time is:  Tower: .0095 sec, +/- .0015 sec; Laptop: .039 sec, +/- .005 sec.  (Timings were computed by averaging six trials, after a discarded warm-up run.)  This is definitely one of those cases where I’m glad I didn’t prematurely optimize, as I would have expected this to be much slower!

We typically install SSDs in all our work machines, but even when people have home machines that use spinning-platter hard drives, startup times are not much slower (likely due to the MFT data staying in cache).

If for a large enough number of entities this timestamp-gathering process becomes too slow, we can always write some code to read the timestamps out of the MFT directly, avoiding all the API call overhead, as in this forum posting.  To date we’ve had no need to do this.

Some people have reported performance problems when Windows has more than 100,000 files in one directory, but this can be worked around: just make 10 directories, with IDs ending with each digit from 0-9 grouped in common directories.  (Why use the least-significant “ending” digit, rather than the most-significant “starting” digit, which might be the impulsive choice?  Because if you do it that way, Benford’s Law says your entities are likely to be unevenly distributed across the directories.  Why not just hash them into a folder?  Because it’s useful to be able to manually navigate to the entity you want, without searching.  (However, partitioning by least-significant digit may be inconvenient if you want to manually edit a sequential range of entities for some reason, because that range will span many directories.  If this is an important use case, perhaps there’s a good compromise like using the tens digit or hundreds digit — though I have the itching feeling that a clearly superior solution would exist.))

As things stand in terms of editing the game world, we are probably not yet at the peak entity count, but there’s unlikely to be more than 3x the number of entities we have now.  Given how low the timing measurements are now, it’s clear that performance will not be a factor in the ability to use this system up until shipping the game.

Partitioning the ID Space

As I mentioned, in our engine, entity IDs are integer serial numbers.  We like this because they are easy to type (we have to type them in the editor sometimes) and at runtime, looking up an entity compiles to a fast indexing of a flat array, with a bounds check.

This introduces another problem for us to deal with in world editing, though: we need to ensure that two users, who are both creating new entities, do not both try to use the same number.  This would result in both users attempting to add completely incompatible .entity_text files with the same name.

One obvious and bulletproof solution is to have the editor connect to the network and atomically grab an ID from a central server each time one is needed.  But again, we don’t want to require users to be on the network in order to edit.

My solution was a somewhat lazy one: we allocate a range of IDs to each computer that edits the world.  These are kept in a file called “id_ranges”; at startup time, the game queries its machine’s MAC address, then looks in the file for the corresponding ID range; if not found, it claims a new range and adds a new line to the file:

 6000 10000 BC-AE-C5-66-28-E9 #Jonathan_office
10000 13000 BC-AE-C5-66-28-48 #Ignacio_office
13000 16000 00-1C-7B-75-00-D8 #Jonathan_laptop
16000 19000 BC-AE-C5-66-28-93 #Shannon_office
19000 22000 BC-AE-C5-56-42-51 #Jonathan_home
22000 25000 C0-3F-0E-AA-F3-F2 #icastano_home
25000 28000 F4-6D-04-1F-D4-36 #Shannon_home
28000 31000 Baker #Baker
31000 34000 BC-AE-C5-56-43-FE #Misc_PC_office
34000 37000 00-E0-81-B0-7A-A9 #eaa
37000 40000 00-21-00-A0-8F-E3 #Orsi
40000 43000 64-99-5D-FD-97-64 #Jonathan
43000 46000 08-86-3B-42-0B-8D #Misc
46000 49000 90-E6-BA-88-9B-B7 #andrewlackey
49000 52000 48-5B-39-8C-80-5E #eaa
52000 55000 50-E5-49-55-64-BF #facaelectrica
55000 58000 00-FF-D4-C7-15-A3 #facaelectrica
58000 70000 Clusters #Clusters
71000 75000 Overflow #Overflow
75000 78000 F4-6D-04-1F-CC-77 #Nacho
78000 81000 BC-AE-C5-74-7B-A3 #Thekla

On each line, everything after the # is just a comment telling us in a human-readable way whose machine is responsible for that range (it defaults to the Windows username, but we can hand-edit that if we want).

(“Baker” is for an automated process, so I made a special case that uses the string “Baker” instead of the MAC address, just to minimize the potential for the build machine to claim a new range without us noticing and failing to check that in; see below.  ”Clusters” is for an editor operation that is also run in an automated process, but which can also be done manually by anyone; cluster generation removes all previous clusters from the world, so if clusters didn’t have their own space, you would end up frequently deleting entities in other peoples’ ranges; this seems like it might be a bad idea, though maybe, in reality, it’s fine and there would be no problem.  But as it stands we know that anything in the Clusters range is completely disposable, so if a conflict ever arises there we can just revert or delete those entities without a second thought.)

This ID space is naturally going to have a lot of big holes in it, but before we ship we will renumber the entities so that we can pack them tightly into an array.  In the meantime, it is sparse but not pathologically so; we can still get away with using a linear array to index the entities, though it will use more memory now than at ship.

This id_ranges file is a bit of a lazy solution, and it causes a few annoyances.  The most annoying thing is that, any time someone runs the editor for the first time on a new machine, they need to immediately check in the id_ranges file to publish their allocation to the rest of the team; and they need to coordinate so that nobody else is trying to do the same thing.  Because the editor just allocates the next 3000 IDs starting with the highest ID in the file, if two people try to allocate IDs simultaneously, they will both claim the same range.  We could solve this by requiring the editor to hold a source control lock to modify the id_ranges file, but I just haven’t gotten around to it, because this hasn’t been a big-enough problem.  Furthermore, we could have the editor automatically commit the changes to the file; but currently, nothing is ever published to the rest of the team automatically — publishing always requires manual action, and this seems like it may be a good thing to preserve.

But, not having implemented these solutions, we have been taken by surprise a couple of times by interactions with a second lazy choice of this system: the fact that we file only records one MAC address per computer.

Most modern computers have multiple MAC addresses (for example: if your computer has a Wifi adapter and two Ethernet ports, it’s probably got at least 3 MAC addresses).  My lazy code just asks Windows to iterate the MAC addresses and reports the first one for use in the file.  Sometimes, though, the reported address will change, which causes someone’s machine to modify the id_ranges file when they didn’t expect it; we had this problem once when an audio contractor, who works offsite, installed some VPN software.  If that user is not vigilant about source control (which non-technical people often aren’t), it can be a while before this problem is discovered; meanwhile, that user has been creating entities that may conflict with someone else’s.

Most or all occurrences of this problem could be solved by recording all of the user’s MAC addresses in the file, instead of one; I just haven’t done so.

If we weren’t using integer serial numbers, we wouldn’t really need id_ranges at all.  For example, if we looked up entities at runtime by hashing unique string identifiers, then we would only need to ensure that each user has a unique string prefix.  We take a slight overkill approach, where each entity ID is a somewhat long string starting with the user’s MAC address; or we could even generate a Windows GUID for each entity.  MAC addresses are 48 bits wide.  If you concatenate the MAC address with a 4-byte integer, that’s just 10 bytes per entity ID, which isn’t much (we currently use 4, already!).  Of course encoding these as strings and storing them in a hash table is going to require more space, but it isn’t ridiculous.  Looking things up in a hash table all the time is slower than indexing a flat array, so we are sticking with the way we do things for now; but this would solve some problems and simplify the overall system, so it sounds like the kind of decision I would change my mind on later, as computers get even faster than they currently are.  So if you implement a scheme like this, you can ignore everything I said about id_ranges.  The biggest problem becomes making it easy to type longer entity IDs into the in-game editor.

Logistical Issues in Editor and File Format Design

In closing, I want to discuss changes we made to the editor and file format, in order to make them compatible with the paradigm of concurrent editing.

Arrays

Just about every game wants to serialize arrays into files.  When transitioning from a binary format, the most natural thing for a performance-minded programmer is to write the array length out first, then write each member of the array.  That way, when reading the file back, we know how much memory to allocate, and we avoid all the slow things one has to do when reading an array of unknown length.  However, encoding arrays in this way results in a textual format that always conflicts, because even when people are making compatible edits to an array, nobody is going to agree on what that array count should be; also, it becomes impossible to manually edit conflicts without typing the count in: if two people add elements to an array, the merged array will contain all the items, but neither version of the file will contain the correct count!

To solve this, of course, we go to the common text-file convention of using a syntactic designation that an array is now over, rather than pre-declaring how many elements there are.  This is obvious, but I mention it here because it is a relatively universal concern.

Automatic Changes

You will also want to minimize or work around cases where the editor automatically changes entity properties without user intervention, because often these will lead to conflicts.  A simple example from our game world: the sun is represented by a few entities that are instantiated in the world, a glowing disc and a couple of particle systems.  Every frame, the gameplay code positions these entities with respect to the current camera transform so that, when the player moves around the world, the sun will seem to be infinitely far away.  This happens in the editor too, because when editing we want to see the world exactly the way it will look in-game.  So, as soon as we enabled concurrent editing, everyone was checking in different coordinates for the sun (depending on where the camera happened to be when they saved!) and this caused a lot of conflicts.  My first solution was to add a hack that zeroes the position of sun entities when saving, but upon loading, this interacted badly with the way particle systems interpolate their positions.  Instead, I added an entity flag that signifies that an entity’s pose doesn’t matter, with respect to thinking about whether it has changed or not, so we will only modify the sun’s .entity_text files if some more-important properties are modified.  (This article is already long, but I’ll say that in general, we detect differences between an entity’s current state and its state when it was loaded, and we only write out .entity_text files when properties have changed.)

Relative Transforms

In our engine, all entities have ‘position’ and ‘orientation’ properties, which always represent the entity’s frame in worldspace, flat and non-hierarchically.  If we want an entity to be attached to another entity’s transform, we set the properties ‘mount_parent_id’, ‘mount_position’, and ‘mount_orientation’ (you can see all these properties in the listing for Door #6555 above); once per frame, the game engine computes the entity’s new worldspace transform based on its mount information, properly handling long dependency chains.  Once we started using textual entity files, we found that entities’ worldspace coordinates would often differ by an LSB or two, even when their parents had not moved (differences especially happening, for example, between Debug and Release builds of the game).  Even without concurrent editing, it’s good to clean this up as a matter of basic hygiene, because otherwise you’re checking in spurious changes all the time, increasing load on the source control system and making it harder for a human to see useful information in the change list.  I chose to solve this problem by making the mount-update code more complex: it detects whether an entity has moved of its own accord, and only updates child entities’ worldspace positions if the parent entity really moved (or if the child entity really moved!).

Terrain Editing

There is one piece of editing workflow that used to make sense before concurrent editing, but which now does not.  We have a terrain system wherein we can move control points around in worldspace, then select a terrain entity, which has a height-field mesh, and press a key to recompute that height field with respect to the new control points.  I originally made this recomputation happen only on manually-selected entities, rather than automatically on the entire world, for UI smoothness purposes: the recomputation takes some time.  But when you only recompute one block of terrain and not its neighbors, you get discontinuities in the terrain; the longer things go on, the more discontinuities you get.  Once in a while I would fix this by selecting all terrain entities in the world, recomputing them simultaneously.

A terrain discontinuity

The catch is that when terrain changes, it affects other entities.  We re-plant grass, so that the grass matches the new elevations; we change the positions of trees and rocks and other objects that are rooted to the terrain at a predetermined height.  Whenever you recompute some terrain, we collect all grass/trees/rocks within a conservative bounding radius, as these are all the things that might possibly change.  When dealing with small groups of terrain blocks, this already is a bit of a recipe for trouble (it is not too hard to overlap someone else’s edits this way); but it also means that if I recompute the entire terrain to eliminate those discontinuities, I am in turn editing all grass, trees, and rocks in the world, and will inevitably conflict with anyone else who has moved any of them. For now, we simply shy away from this kind of global operation; when you get a conflict in some entity that you probably didn’t care about, like grass, it is easy just to accept the remote version of the file.  Long-term, I expect some kind of heavier-weight solution would have to happen here, though I hope it does not involve much locking.

Issuing Source Control Commands

Our editor automatically issues source control commands when entities are created or destroyed, because it would be extremely tedious and error-prone to expect users to do this.  Initially we had some problems with source control commands unexpectedly failing, which quickly leads to chaos.  After some work, issuing source control commands seems to work with full consistency, but we are still a little wary of it; so we don’t, for example, want to issue the commands via a background process, adding more complexity to the system.  The problem with this is that svn commands are a bit slow.  Sometimes when I have been editing the world for a while, it can take 15 to 30 seconds to save my changes, which would have happened instantaneously without the svn commands.

Conclusion

We’ve put together a system that lets us concurrently edit an open world, using source control to manage our world edits.  It works well enough for us to ship the game, so I am happy with it.  If the scheme ever fails in some way, or requires heavy modification, I will post an update!

When we sat down to start working with the architects, one of the first things we did was put together a history of the island where The Witness takes place: how long has the island been there?  What groups of people lived there, and what kind of structures did they build?  How did later civilizations use the structures built by earlier groups?

Quickly we developed a running theme of sites that are old and have over time been used in two or three different ways.  When entering a new location, you probably first notice the most basic elements of its structure: where you can walk to easily, versus where you can’t; what puzzles or goals are calling for your attention, and what walls or doors might be preventing you from getting to them.  If this is all you care about, you can then play through the area in a very utilitarian fashion.  But if you have an eye for detail, you may notice the elements of modern construction shoring up the design of an older structure with a different purpose.  The design of the new structure tells you something about what the newer guys were doing there, and how it differs from what the older guys were doing there.  And if you look really closely, you can see traces of the original footprint of the site (perhaps these are very subtle, some bricks just rising above the dirt in a few places), and infer the purpose of this very old site too.

If you see the different civilizations that came to this island as embodying different philosophies; and you see the structures they built as representative of the way these philosophies led them to interact with the world; and you see further that when they replaced a site, it represents the rejection of some older worldview that they consider no longer useful, then perhaps you start to get some idea of the amount of backstory that can be encoded into the world, nonverbally.

All of this backstory is also directly relevant to the main story — who you are, why you are on this island, how you may get back home.  So, to put it another way, the game is constructed so that the more you pay attention to tiny details during your travels, the more insight you will have to the central story, even though it may not be obvious at any given time what a particular detail has to do with that story.

The most-recent issue of Game Informer printed an article that showed some of the first screenshots of the newly-architected areas appearing in the game engine.  It is an interesting article; pick up the issue if you haven’t!  They also put up a companion article online (but which not as extensive) here:

(Summary article at Game Informer)

The area featured in this article is the one that we have taken furthest so far in terms of modeling and texturing.  Here are some other shots of it:

The Keep, before and after

The Keep, close-up 1

The Keep, close-up 2

The Keep, close-up 3

We’ve done basic planning for some of the other areas, so I thought it would be nice to show some screenshots of work in progress.  For each of these, there’s a ‘before’ version (when we had a fully-playable game, but basic placeholder structures) and after (once we have incorporated the architects’ designs, revised the site plans and in some cases even the puzzles, etc):

 

The Compound, before and after

Glass Factory, before and after

Peninsula, before and after

Vault, before and after

Don’t regard these “after” images as representative of what the game will look like!  They are still works in progress, and the final game will look substantially better: scenes like these will feel richer due to the placement of detail objects, further care put into the texture mapping and lighting, refinement of the still-rough-drafty terrain features around these structures, and many other bits of subtle game development magic.  But I wanted to show these now since they provide a good taste of the game’s flavor, and they show how much structural intelligence is brought to the table by working with the architects.  Yes, we could have started with the placeholder structures and made them more elaborate and better-looking, in a general video-game-level-design way, but that’s different from having well-thought-out ideas subtly embodied in the structures of the areas, which is what we are going for.

Click here to view the video on YouTube.

Q.U.B.E. will be available on Steam.

(Web site for the game)

(Indie Fund announcement page)

Eventually, these things have to be figured out. Over the past month or so we’ve done some of that figuring. We nailed down the architecture for a couple of areas (including the all-important area where the player starts the game). We’ve done this in a way where the architecture and the puzzle ideas have synergy, which is always nice. In particular, there was one location that, from the beginning of development, was basically a big box with a window cut into it and some puzzles stuffed inside. For a long time (maybe 2 years!) I was never sure what to do with it, but eventually the architects suggested an idea for that location. I mocked it up and didn’t like it. They suggested another one. This one sparked an idea for how to make the structural properties resonate with the puzzle theme, and when I built even just a rough-draft of the building, I found the result to be strong. Lately we’ve been finalizing that location too.

So anyway, there are all these spots on the island that represented vague ideas, with puzzles there that were just in rough form, or that weren’t even implemented fully, but just served as reminders to do something there. I’ve been going through and filling in a bunch of those.

I have also been working with some other puzzles, which seemed “done enough to ship”, and thinking hard about how to make them better. Braid benefited from a lot of this kind of attention — a lot of games would have shipped with just early versions of the puzzles in that game, considering them to be just fine; but the more I worked with the puzzles in Braid, the more I found better, simpler, more-to-the-point versions of them. The same thing is happening here (though The Witness is a much bigger, more-complicated game, so sometimes it is challenging to get to that same kind of clarity.)

With regard to the visuals (modeling and texturing), things have been coming along nicely as well. Pretty soon a high-profile game magazine will publish some screenshots of one of the most-finished areas of the game, showing the look we are targeting and giving a strong hint of what the game will be like stylistically. When the magazine is out there we will link it here.

(This has not been a particularly coherent update; just wanted to let you all know what is up!)

Any system of interactivity can of course be explored: If X happens, what are the consequences? What are all the ways in which pattern Y expresses itself, and to what do those expressions lead? By inspecting the structure of a system in this way, we can find the core ideas of the system, and see how those ideas illustrate fundamental truths of our universe. We present a game design aesthetic that values looking for systems that express these truths in the cleanest possible way. We explain how this is different from more-traditional combinatoric design techniques; we show examples from our games and describe a method for applying the aesthetic in general.

Here’s a link to the Indiecade page, on which there’s also a presentation by Richard LeMarchand, and more to come.

Thanks very much to Ida C. Benedetto for recording the talk!

I’ll post something more concrete before too long, but I wanted to at least say something, since it’s been a while.

Team-wise, we have just hired two people (a 3D artist and a programmer), and we plan to hire another programmer as well, once his visa clears and he can get into the country. This is a pretty big staffing-up, percentage-wise, and will allow us to make things quite nice. The new folks will be starting over the next couple of months (they need to wind down the things they are working on currently!)

The current puzzle count is 350.

Some parts of the island are pretty messed up in this screenshot. For the press tour, we made sure the whole game was more-or-less solid and playable. Just after that, though, we were free to tear things apart, move things around, change the sizes of areas, add new areas, and all that. We’re still in the middle of that process and not everything has been made playable again.

Here’s the link.

The difference is not too pronounced in this scene, but with smoother-colored textures it becomes a lot more obvious (and I do want to use more smoother-colored textures, art-direction-wise!)

The situation is, the lightmaps are HDR (for some definition of “high”), so we needed a way to encode those values reasonably into a bitmap. We were using RGBM where M is just an overall brightness factor; you read RGB from the texture then multiply by M to get the output color. We used to just encode this naively, but the problem is, when M is less than 1 then it actually kills precision in your RGB, so the solution is, well, don’t ever let M go below 1; just clamp it. This is not rocket science and in fact is the kind of thing that is a little funny, as one starts wondering why we didn’t think of that a long time ago, but hey, that’s how it is sometimes. The results made me happy, though, so I am posting them.

The improved image quality may enable us to have higher dynamic range throughout the game (this is probably something we will evaluate later on), which would be very nice!

Previews of the game are now being published, and readers of this blog have been asking for gameplay details for a while. So it seems like a good idea to list the previews here. I’ll keep updating this post as I find more.

on Kotaku, by Stephen Totilo
on Joystiq, by Ben Gilbert
on Gamespot, by Sophia Tong
on IGN, by Anthony Gallegos
on Gamasutra, by Christian Nutt
on 1UP, by Matt Leone
on GamesRadar, by Charlie Barratt
on Giant Bomb, by Ryan Davis
on Giant Bomb, by Patrick Klepek (interview-ish)
on Time Techland, by Evan Narcisse
on IFC, by Evan Narcisse (interview)
by Dan Griliopoulos on Rock, Paper, Shotgun

interview with GamesRadar
interview with Cory Cohen of OXM
long interview with Dan Griliopoulos of Rock, Paper, Shotgun (contains a bunch of stuff not elsewhere)

interview with Christian Donlan of Eurogamer

(Sometimes these previews, from the way they are worded, make it seem like I made the entire game with my own bare hands. That is not the case; it’s just how this kind of article tends to sound. There are a number of people working on this game, who have creative input in various ways; see the About page for a partial list!)

• Lightmap Parameterization
• Hemicube Rendering and Integration

Unfortunately, by the time I was done with these, the work I had done on the gradients was not so fresh anymore, which is why it has taken me so long to complete this article.

In my previous post I mentioned the need to find an algorithm that would allow us to extrapolate the irradiance to fill lightmap texels with invalid samples, to smooth the resulting lightmaps to reduce noise, and to sample the irradiance at a lower frequency to improve performance. As some avid readers suggested Irradiance Caching solves all these problems. However, at the time I didn’t know that, and my initial attempts at a solution were misguided.

Texture Space Approaches

I had some experience writing baker tools to capture normal and displacement maps from high resolution meshes. In that setting it is common to use various image filters to fill holes caused by sampling artifacts and extend sampled attributes beyond the boundaries of the mesh parameterization. Hugo Elias’ popular article also proposes a texture space hierarchical sampling method, and that persuaded me that working in texture space would be a good idea.

However, in practice texture space methods had many problems. Extrapolation filters only provide accurate estimates close to chart boundaries, and even then, they only use information from one side of the boundary, which usually results in seams. Also, the effectiveness of irregular sampling and interpolation approaches in texture space is greatly reduced by texture discontinuities and did not reduce the number of samples as much as desired.

After trying various approaches, it became clear that the solution was to work in object space instead. I was about to implement my own solution, when I learned that this is what irradiance caching was about.

Irradiance Caching

This technique has such a terrible name that I would have never learned about it if it wasn’t due to a friend’s suggestion. I think that a much more appropriate name would be adaptive irradiance sampling or irradiance interpolation. In fact, in the paper that introduced the technique for the first time Greg Ward referred to it as lazy irradiance evaluation, which seems to me a much more appropriate name.

There’s plenty of literature on the subject and I don’t want to repeat what you can learn elsewhere. So, in this article I’m only going to provide a brief overview, focus on the unique aspects of our implementation, and point you to other sources for further reference.

The basic idea behind irradiance caching is to sample the irradiance adaptively based on a metric that predicts changes in the irradiance due to proximity to occluders and reflectors. This metric is based on the split sphere model, which basically models the worst possible illumination conditions in order to predict the necessary distance between irradiance samples based on their distance to the surrounding geometry.

Interpolation between these samples is then performed using radial basis functions, the radius associated to each sample is also based on this distance. Whenever the contribution of the nearby samples falls below a certain threshold a new irradiance record needs to be generated, in our case this is done by rendering an hemicube at that location. In order to perform interpolation efficiently the irradiance records are inserted in an octree, which allows us to efficiently query the nearby records at any given point.

For more details, the Siggraph ’08 course on the topic is probably the most comprehensive reference. There’s also a book based on the materials of the course; it’s slightly augmented, but in my opinion it does not add much value to the freely available material.

The Physically Based Rendering book by Matt Pharr & Greg Humphreys is an excellent resource and the accompanying source code contains a good, albeit basic implementation.

Finally, Rory Driscoll provides a good introduction from a programmer’s point of view: part 1, part 2.

Our implementation is fairly standard, there are however a two main differences: First, we are sampling the irradiance using hemicubes instead of a stratified Montecarlo distribution. As a result, the way the we compute the irradiance gradients for interpolation is somewhat different. Second, we use a record placement strategy more suited to our setting. In part 1 I’ll focus on the estimation of the irradiance gradients, while in part 2 I’ll write about our record placement strategies.

Irradiance Gradients

When interpolating our discrete irradiance samples using radial basis functions the resulting images usually have spotty or smudgy appearance. This can be mitigated by increasing the number of samples and increasing the interpolation threshold, which effectively makes them overlap and smoothes out the result. However, this also increases the number of samples significantly, which defeats the purpose of using irradiance caching.

One of the most effective approaches to improve the interpolation is estimating the irradiance gradients at the sampling points, which basically tell how the irradiance changes when the position and orientation of the surface changes. We cannot evaluate the irradiance gradients exactly, but we can find reasonable approximations with the information that we already have from rendering an hemicube.

In my previous article I explained that the irradiance integral was just a weighted sum over the radiance samples, where the contribution of each sample is the product of the radiance through each of the hemicube texels , the solid angle of the texel , and the cosine term . That is, the irradiance at a point is approximated as:

The irradiance gradients consider how each these terms change under infinitesimal rotations and translations and can be obtained by differentiating .

The radiance term is considered to be constant, so the equation reduces to:

In reality, the radiance is not really constant, but the goal is to estimate the gradients without using any information in addition to what we have already obtained by rendering the scene on a hemicube. So that we can improve the quality of the interpolation by only doing some additional computations while integrating the hemicube.

Rotation Gradient

The rotation gradient expresses how the irradiance changes as the hemicube is rotated in any direction:

The most important observation is that as the hemicube rotates, the solid angle of the texels with respect to the origin remains constant. So, the only factor that influences the gradient is the cosine term:

With this in mind, the rotation gradient is simply the rotation axis scaled by the rate of change, that is, the angle gradient:

And the rotation axis is given by the cross product of the z axis and the texel direction :

Therefore:

This can be simplified further by noting that is the length of before normalization and that results in a very simple expression:

Finally, the rotation gradient for the hemisphere is the sum of the gradients corresponding to each texel.

The resulting code is trivial:

foreach texel, color in hemicube {
    Vector2 v = texel.solid_angle * Vector2(texel.dir.y, -texel.dir.x);
    rotation_gradient[0] += v * color.x;
    rotation_gradient[1] += v * color.y;
    rotation_gradient[2] += v * color.z;
}

Translation Gradients

The derivation of the translation gradients is a bit more complicated than the rotation gradients because the solid angle term is not invariant under translations anymore.

My first approach was to simplify the expression of the solid angle term by using an approximation that is easier to differentiate. Instead of using the exact texel solid angle like I had been doing so far, I approximated it by the differential solid angle scaled by the constant area of the texel, which is a reasonable approximation if the texels are sufficiently small.

In general, the differential solid angle in a given direction is given by:

For the particular case of the texels of the top face of the hemicube, n is equal to the z axis. So, the differential solid angle is:

We also know that:

which simplifies the expression further:

Using this result and factoring the area of the texels out of the sum we obtain the following expression for the gradient:

The remaining faces have similar expressions, slightly more complex, but still easy to differentiate along the x and y directions. We could continue along this path and we would obtain closed formulas that can be used to estimate the translation gradients. However, this simple approach did not produce satisfactory results. The problem is that these gradients assume that the only thing that changes under translation is the projected solid angle of the texels, but that ignores the parallax effect and the occlusion between neighboring texels. That is, samples that correspond to objects that are close to the origin of the hemicube should move faster than those that that are farther, and as they move they may occlude nearby samples.

As you can see in the following lightmap, the use of these gradients resulted in smoother results in areas that are away from occluders, but produced incorrect results whenever the occluders are nearby.

Left: Reference. Middle: Irradiance caching without gradients. Right: Irradiance caching with flawed gradients

The hexagonal shape corresponds to the ceiling of a room that has walls along its sides and a window through which the light enters. The image on the left is a reference lightmap with one hemicube sample per pixel. The image on the middle is the same lightmap computed using irradiance caching taking only 2% of the samples. Note that the number of samples is artificially low to emphasize the artifacts. On the right side the same lightmap is computed using the proposed gradients. If you look closely you can notice that on the interior the lightmap looks smoother, but close to the walls the artifacts remain.

This is exactly the same flaw of the gradients that Krivanek et al proposed in Radiance Caching for Efficient Global Illumination Computation, but was later corrected in the followup paper Improved Radiance Gradient Computation by approaching the problem the same way Greg Ward did in the original original Irradiance Gradients paper.

I then tried follow the same approach. The basic idea is to consider the cosine term invariant under translation so that only the area term needs to be differentiated, and to express the gradient of the cell area in terms of the marginal area changes between adjacent cells. The most important observation is that the motion of the boundary between cells is always determined by the closest sample, so this approach takes occlusion into account.

Unfortunately we cannot directly use Greg Ward’s gradients, because they are closely tied to the stratified Montecarlo distribution and we are using a hemicube sampling distribution. However, we can still apply the same methodology to arrive to a formulation of the gradients that we can use in our setting.

In our case, the cells are the hemicube texels projected onto the hemisphere and the cell area is the texel solid angle. To determine the translation gradients of the texel’s solid angle is equivalent to computing the sum of the marginal gradients corresponding to each of the texel walls.

These marginal gradients are the wall normals projected onto the translation plane, scaled by the length of the wall, and multiplied by the rate of motion of the wall in that direction.

Since the hemicube uses a gnomonic projection, where lines in the plane map to great circles in the sphere, the texel boundaries of the hemicubes map to great arcs in the hemisphere. So, the length of these arcs is very easy to compute. Given the direction of the two texel corners , the length of their hemispherical projection is:

The remaining problem is to compute the rate of motion of the wall, which as Greg Ward noted has to be proportional to the minimum distance of the two samples.

The key observation is that we can classify all the hemicube edges in wo categories: Edges with constant longitude ( is invariant), and edges with constant latitude ( is invariant).

In each of these cases we can estimate the rate of change of edges with respect to the normal direction by considering the analogous problem in the canonical hemispherical parametrization:

For longitudinal edges we can consider how changes with respect to motion along the x axis:

and for latitudinal edges we can consider how changes with respect to motion along the y axis:

As we noted before, the rate of the motion is dominated by the nearest edge, so takes the value of the minimum depth of the two samples adjacent to the edge.

A detailed derivation for these formulas can be found in Wojciech Jarosz’s dissertation, which is now publicly available online. His detailed explanation is very didactic and I highly recommend reading it to get a better understanding of how these formulas are obtained.

Note that these are just the same formulas used by Greg Ward and others. In practice, the only thing that changes is the layout of the cells and the length of the walls between them.

As can be seen in the following picture, the new gradients produce much better results. Note that the appearance on the interior of the lightmap is practically the same, but closer to the walls the results are now much smoother. Keep in mind that artifacts are still present, because the number of samples is artificially low.

Left: No gradients. Middle: Flawed translation gradients. Right: Improved translation gradients.

In order to estimate the translation gradients efficiently we precompute the direction and magnitude of the marginal gradients corresponding to each texel edge without taking the distance to the edge into account:

// Compute the length of the edge walls multiplied by the rate of motion
// with respect to motion in the normal direction
foreach edge in hemicube {
    float wall_length = acos(dot(d0, d1));
    Vector2 projected_wall_normal = normalize(cross(d0, d1).xy);
    float motion_rate = (constant_altitude(d0,d1) ?
        -cos_theta(d0,d1) :      // d(theta)/dx = -cos(theta) / depth
        -1 / sin_theta(d0,d1);   // d(phi)/dy = -1 / (sin(theta) * depth)
 
    edge.translation_gradient = projected_wall_normal * wall_length * motion_rate;
}

Then, during integration we split the computations in two steps. First, we accumulate the marginal gradients of each of the edges scaled by the minimum edge distance:

foreach edge in hemicube {
    float depth0 = depth(edge.x0, edge.y0);
    float depth1 = depth(edge.x1, edge.y1);
    float min_depth = min(depth0, depth1);
 
    texel[edge.x0, edge.y0].translation_gradient -= edge.gradient / min_depth;
    texel[edge.x1, edge.y1].translation_gradient += edge.gradient / min_depth;
}

Once we have the per texel gradient, we compute the final irradiance gradient as the sum of the texel gradients weighted by the texel colors:

foreach texel, color in hemicube {
    Vector2 translation_gradient = texel.translation_gradient * texel.clamped_cosine;
    translation_gradient_sum[0] += translation_gradient * color.x;
    translation_gradient_sum[1] += translation_gradient * color.y;
    translation_gradient_sum[2] += translation_gradient * color.z;
}

Conclusions

Irradiance Caching provides a very significant speedup to our lightmap baker. On typical meshes we only need to render about 10 to 20% of the samples to approximate the irradiance at very high quality, and for lower quality results or fast previews we can get away with as few as 3% of the samples.

The use of irradiance gradients for interpolation allows reducing the number of samples significantly, or for the same number of samples produces much higher quality results. The following picture compares the results without irradiance gradients and with them, in both cases the number of samples and their location is the same, note how the picture at the bottom has much smoother lightmaps.

I think that without irradiance caching the hemicube rendering approach to global illumination would have not been practical. That said, implementing irradiance caching robustly is tricky and a lot of tuning and tweaking is required in order to obtain good results.

Note: This article is also published on #AltDevBlogADay.

Here’s some screenshots I just took. Here is a group of trees without a lightmap. As you can see, the underside looks pretty flat and boring, and it’s hard to see which limbs are in front of others.

Here’s the lightmap just by itself. While you can see a few lighting artifacts (like where the cross trees cross), overall the trees look pretty good even without diffuse textures.

And here’s the final. As you can see, the underside has a lot more definition to it.

Once I get some LOD’s I like, I’ll write up another update.

Sun beams. These are accomplished with a radial blur centered on the sun, during postprocessing. The nice thing about this effect is how dynamic it is — as you walk beneath the trees, the shape and intensity of the beams are changing all the time.

Due to the way the effect works, sometimes beams are drawn that are a little bit unphysical. You probably have to be looking closely to see that, but I am a little bit uncomfortable with it given that the game is all about the opposite of that. We’ll see how this evolves. We’ve been talking about augmenting this with other effects.

A revised water reflection effect.

Softening of the shoreline, so that it doesn’t look like one big alpha-blended water polygon intersecting the ground.

Here’s what the island looks like now:

It’s fun to look at this compared to another shot of the island, say, from about a year ago:

The current puzzle count is 328.

The talk’s about 38 minutes long, with about 40 minutes of Q&A afterward. Here’s a slideshow:

CSUA talk

If the above embedded player doesn’t work for you, or you prefer to download the files directly, here they are:

audio track
slides (ppt format)

I’ve worked on trees for games over the years, and I’ve come to a conclusion: There’s no good way to make them currently. Trees reject the idea of low poly versions of themselves by their very nature, and what you’re left with is many unsatisfactory ways of making them. It’s up to the artist to pick which one of these techniques to use and to do their best to make it work. That choice depends on a lot of things. Does it have to be real-time, or can it be pre-rendered? Where will the player be when he sees the tree? Up close, far away, underneath, from above, or (in the worst case) all of these? Will there be a few trees or a forest of them? Then there’s the engine constraints.  Overdraw, poly counts, texture space, LOD systems, lighting and light maps; all these affect how the trees need to be created. And none of that is even scratching the surface on the style you want the trees to have.

Still, I like a challenge as much as the next person.

Here’s an early visual mockup of the game that contains some trees. I didn’t actually make this, but it did give me some direction is where to go. As you can see, the trees are very colorful and full of volume. I wasn’t sure how close I could get to this, but I was going to give it a try. The problem with most low-poly game trees is that they are made with large sheets of polygons with branch and bough textures on them…and while this is certainly a way of making game trees, they usually can’t be considered volumetric.

The current trees I have right now started from a lucky accident. Early on the engine couldn’t handle alpha maps, so I want to make a bough structure that would look vaguely leafy without being too many polys and without using opacity maps. That’s how I came up with this.

It’s basically a ton of triangles sitting in a bowl. The bowl is there to simplify the bottom and hide the triangle nature of the triangles. It worked fine for the time I needed it. Recently I started adding a bunch of trees to the world and I wanted something fairly lightweight for even the large trees (the one I had done just before that was 40,000 polys, which was way too big). So I went to the old non-opacity tree, added some opacity maps and tried it out. And, surprisingly, it worked really well. It was giving me a volume I didn’t expect, and it looked really good at a distance.

Not only that, I took a look at them with only the lightmaps on and the structure looked really nice…especially the branches underneath.

They were still too heavy though, being 12 thousand polys. Worse, they looked really bad up close and you could see that, what read as a nice, thick foliage from a distance, was really a mess of triangles.

So the past few days I’ve been trying to make the trees look better from a close distance and be lower poly without losing what I liked about the previous ones. After a bit of struggle, I got something that seems to do the trick. Ignacio helped a lot, making a shader that made polys disappear when they became too edge on to the camera, which eliminated many ugly artifacts. The foliage is half the polys of the other trees, plus it looks a lot better close up. Here’s a close shot of it, plus a comparison shot with an older tree.

I still have a bunch of other ideas I want to add to these to make them look better, but I think we’re on the right track to getting trees that really do have a nice amount of volume and harken back to the early mockups.

But we also figured, hey, why not play around with the color of sunlight as well? The idea is that as you walk around in the game, the color of the sun and ambient light will interpolate as you move. We haven’t hooked it up that way yet; what you see below is just done by tweaking global variables in the game. Well be doing more experiments in this direction…

Much denser and more visually interesting; it’s starting to look like maybe a good game!

Here are some close-ups of different areas now.

Autumnal:

Desert:

Foresty (there is a lightmapping problem on some of these trees so they are brighter than they should be!):

But now we are thinking about how to create colorful scenes, like in the previous post, and this style of blend-mapping did not seem like a very good idea. In order to create gradients across the scene, we would need to have a lot of blend layers; this would be slow and use a lot of texture memory (since now we are storing a lot of individual blend maps). Even once we did that, it would be laborious to control, and it’s unclear that it would look good.

Ignacio mentioned that the Wolfire guys were just dropping one color map across their whole terrain. That seemed like a good idea to me — I have done that before, but in contexts where we were trying to really crank up the resolution to levels that were ridiculous at the time, like 8192×8192 texture maps back in 2001. Rather than doing that, the idea now is to just use what is a reasonable-sized texture on a modern machine, and see what it looks like. So Shannon painted a 2048×2048 texture of the island (of which we are only really using less than half right now) and here’s how it looks from up high:

Immediately this strikes me as much nicer, and much more in-tune with the art direction we want for the game. The buildings and other meshes now stick out like sore thumbs, but that is just what happens when you improve only a part of the scene. We’ll be working on this.

Here’s a scene from closer in:

As you can see, we have less detail close-up on the ground; things look a bit blurry. However, the colors are a lot better, and I think if we can adapt this scheme into something that permits sharper details up close, we will be able to make something that has a lot of character and looks special. In these shots, we are still using the old blend map system to tile high-detail normal maps across the terrain; that helps, but by itself it’s not enough. (In recent videos for Overgrowth, like this one, you can see that they are doing something that provides close-up details, maybe just intensity maps that are being modulated by the color map). So we’ll keep working on this and see how it goes.

Meanwhile, Ignacio has been revising the shadow map rendering. We used to have very speckly shadow maps, as I was using a 4-sample Poisson disc to sample the shadow map. Ignacio recently switched this to a Gaussian that has more taps, then applied some math tricks to reduce the number of taps. This results in softer shadows:

At some angles, shadow map jaggies still show up pretty clearly:

This is something we might work around in the final game by changing the scene when this happens! Or, it’s possible the shadows will be modified further so that they look smoother in cases like this. Who can say?

Shannon did a few color studies by taking screenshots of the game world as it stands today, then painting over them to build scenes with different palettes. Here’s what he came up with:

I think it looks quite nice. Don’t assume much based on the painterly style of these images; they were intentionally done at a low level of detail, so that we could quickly get a feel for the colors. We don’t know yet exactly what the rendering style of the game will be in the end (but that is something we will start working on soon!)

The puzzle count is now 285.

This means I am getting closer to my Gameplay Complete milestone. The bulk of the game design is now more-or-less there, in the sense that you can go and solve all the puzzles and they are basically the body of puzzles that will be there for the final game (though in the final game they will be refined, and more will be added — but if we had to ship with only the puzzles that are in there now, we could do that, and it would be a good game). However, there’s no endgame or winning condition, so I will be working on this next.

I think the endgame is going to be a bunch of work — the original idea was ambitious, and I need to do a bunch of redesign on it since the game as a whole has evolved since the time the endgame was originally conceived. We’ll see how it goes.

The visuals are still placeholder. In a couple of weeks we have some architects coming in to start designing buildings and terrain; at that time our production pipeline will change and we’ll be moving a little closer to “the real visuals”.

See the video here.

Thanks to blog commenter justin for pointing this out!

As a result of these sessions it immediately became clear how to redesign some of the areas to improve pacing. So I started radically restructuring three of the major areas, as well as rearranging the relative positions of areas on the map. That process isn’t done yet, but it is starting to converge (just one more area left to redesign). I am very happy with the two areas already redesigned; they are obviously far better than the original versions. It feels very rewarding when you feel good about a game already, and then do something to improve it so much.

Also, I’ve shrunk the island again. Here’s what it looks like now:

The current puzzle count is 275.

Also, we are hiring.

We at Thekla, Inc. are a small independent developer making games with tradition-breaking designs.  We are pushing the boundaries of what a video game can be. Our current project is The Witness, a puzzle/exploration game designed by Jonathan Blow, designer/programmer of the critically-acclaimed Braid.  Our office is in Berkeley, California, USA but we will consider remote work for the right candidates.

Senior Game Programmer (full-time)

Work with other excellent programmers to build technically-sophisticated games for a variety of platforms, including consoles, PCs, and tablets.  Work on systems-level infrastructure as well as gameplay code.  You get a lot of freedom to approach problems the way you want to, and to employ your full range of technical creativity.  Management overhead is minimal.  There are many perks you won’t find in other jobs.

Our code is mainly in C++.  When it comes to building software, we have a no-nonsense approach.

Job Requirements:

• Be good at making video games.

Senior Game Programmer (part-time, contract)

Like the job above, but part-time.

To inquire about any of these jobs, please send email to jon@number-none.com.

Here’s what it looks like now:

It does not look shipping-quality, but then, almost nothing in the game does! As the first results obtained by just slapping some pre-existing grass meshes down on the terrain without trying too hard, I find it highly encouraging.

Note that grass doesn’t cast shadows right now. There are two reasons for that: rendering speed and aliasing of the shadow map. I suspect we will try to make grass cast shadows before we ship, though, and if we make something good happen there, I think it will really boost the visual quality.

The one piece of code that pushed the grass over the threshold into general usability is that it now has the ability to sample terrain blend maps. (The terrain uses a blend shader, where a non-visible blend texture map controls the degree of mixing of different terrain materials. This is becoming a common thing in 3D games; Uncharted 2 and Crysis 2 do it, among others.) The grass can now peek at the blend map and reduce its density in areas of particular blend values, which is why in this screenshot that path going up the hill is free of grass. This tremendously reduces the amount of manual tweaking required to make the grass not do obviously-bad things.

However, the grass doesn’t check for buildings yet, which means if you place a field of grass across the area of a building, the grass will probably stick up through the floor inside the building! Fixing that is next on my to-do list, but it involves a lot of raycasting, so hopefully it doesn’t make grass placement too slow.

In the future I plan to try using this same “grass” system for other kinds of scattered ground objects (fallen leaves, twigs, small rocks, etc).

Toward the back you can see some of the stuff that was being worked on in the previous post (but from a different angle!)

The current puzzle count is 251. That’s pretty good, but on top of the increase from last time, another 25 of the old puzzles were rewritten. So a bunch of stuff has gotten done!

Over the next couple of weeks, I will be hammering on the puzzles even more, as we get the game to a new level of completion and playability.

I was talking to David Hellman about the game the other day, and he was asking when the island was going to become less like a flat plane where you can just walk to any destination from anywhere else.

It’s been on my to-do list for a while to add more topology to the island. That week I had been working on an area full of puzzles, and I wanted the buildings that define this area to have more of a purpose than they currently do; a topological purpose seemed like the right thing.

So I figured I’d add a canyon, and the building containing these puzzles would also serve as a bridge across the canyon.

The natural way to build the canyon in this engine is to shape it out generally with terrain control points, and then make the walls steep by putting mesh objects in there (so that you can’t walk up them, and so that the walls of the canyon won’t look super-distorted the way heightfield terrains usually do when there are steep slopes.)

Here was the first cut at it:

As you can see, there are a lot of weird ripples and curves happening there. This is what signal processing people might call “ringing”. What’s happening is that the Kriging system that we are using for terrain modeling is always trying to reconstruct a smooth function, but what we are trying to build here is a discontinuous function (or else a function so steep that it has a lot of high frequencies in it).

There are a number of ways that one might fix this, but the cheap and hacky one, which would work not only for Kriging but for any other terrain modeling stuff we might stick in in the future, is to add the notion of separating planes that represent discontinuities in the terrain. When evaluating the height of the terrain at any point, we just ignore any control points that are on the other side of the separating plane.

Adding discontinuities has been on my to-do list since last year when Kriging was added to the engine. It was pretty obvious that it would be needed eventually, but in the spirit of getting things done and not rat-holing on technology, I chose to put it off for as long as possible. But it makes sense that if we are starting to add more topology to the island, we are going to want to have more steep places, and so it’s time to add this.

(We’ve already had this steepeness problem before, though: if you look at this plateau that is near the camera in our regular island snapshots, you’ll see that the whole thing is made out of mesh objects, rather than terrain; we did this in order to avoid this very same ringing problem:)

It didn’t take long to add discontinuity entities into the editor; here’s how they are represented:

To start with, I just used a really dumb heuristic to decide when control points should be excluded. When we use this, here’s the result:

There is still a bunch of heightfield terrain sticking out. What’s going on? If we look back at our red discontinuity planes, it is clear that there are some terrain sample points inside the canyon that are getting treated as if they were outside:

What’s going on is that my simple criterion said “if you are on the opposite side of a discontinuity plane from a control point, ignore that control point”. The problem is that I am treating them as infinite planes, so those points that you see, which are in front of one plane, are actually outside a neighboring plane, if you were to extend that plane infinitely. This is no surprise; I knew it would be a problem, but I am always about implementing the simple thing first and seeing what happens. But now I need a heuristic that will fix this problem.

One solution would be to combine the discontinuity objects into sequences of points, and then do some more-heuristicky things that may be easier because we have an ordering between the points.  But I don’t want to resort to that if I don’t have to, because that may make the system less general, and may make it harder to edit.  (Then again it may make it easier to edit, because we have editor support for paths!  But if we want to, we can easily go the other way, from a path down to individual slabs, so it’s better to take the most-primitive thing as input if we are able to.)

I added a check that would make discontinuity planes irrelevant if the sample points are closer to the side planes of these very thin slabs than they are to the face planes.  In theory that would at least help, but upon running this, the scene looked almost exactly the same.

Then upon further investigation I found that I was computing totally wrong planes in the Kriging code; I was transforming the coordinates incorrectly, and so the planes were not in fact where the red slabs in the picture would show them to be.  I fixed this:

Ahhh, much better.  This seems like it will be good enough for now.  There’s still the question of how to seamlessly blend the heightfield terrain into the mesh objects, but that can wait until closer to ship.  For now, it doesn’t need to be pretty; it just needs to let us build things of roughly the right shape.

The heuristic I use for discarding points is still pretty simplistic and may cause problems.  My hope is that most of those problems can be worked around just by changing the planes a bit.  If that turns out not to be true, a little more work will need to be done here!

So, there are 222 puzzles left in the game (I just deleted about 10% of them).

Many more puzzles are going to be added; I would guess that by the time the game is done, there will be over 400. We’ll see; I think this will be a fun statistic to watch, along with those screenshots of the island.

In comparison, Braid has about 76 puzzles in it. So that tells you something about the relative sizes of the games. That said, the puzzles in The Witness are often intended to be bite-sized and quick, so it’s not really a direct comparison. (On the other hand, some of the puzzles are most definitely not bite-sized and quick!)

The past week has been a very productive time in terms of world-building. The island has been shrunk further, with several of the locations that used to be further to the right moved inward. The white-walled location from last time used to be a standalone structure, but is now built into the side of that plateau nearest the camera, and has a new structure next to it. One of the prominent garden areas has been redesigned; you may notice it a bit to the left of the center of the map. On top of all this, a new underground area has been added, and I really like some of the puzzles in there.

When the new area gets finished gameplay-wise I will post some shots of that.

Up next: adding more puzzles!

The most obvious difference from the previous shot is the thicker atmosphere. Ignacio changed the fog model; the fact that we have a bit more fog, combined with the art style we’ve converged on, should help us do high-quality LOD handling for distant objects.

Your bonus picture for today is part of an indoor complex that I am very happy with. This room is all white with a bit of colored glass. Note the lovely lightmapping, even on such slapped-together geometry:

The general requirement was that we need to be able to capture a pre-rendered scene with a really wide viewing angle. At first I was confused about the technical aspects of the problem (hopefully forgivable, since I spend most of my time thinking about gameplay these days, so my tech is a little rusty), so I thought that we might need to have a linear projection in order to solve the specific problem under consideration. So Shannon made a mock-up scene in 3D Studio MAX and we started making prerenders with increasingly wide camera angles, in order to test our special environment mapping. Here are three shots of the same scene with fields-of-view of 90 degrees, 120 degrees, and 150 degrees:

(Click on the images to see actual sizes.)

It’s clear that as the field of view becomes wider, we can see more of the scene at once, laterally. This is good for our application, because we want to see as much as possible! But at the same time, when the angle is wide, objects at the center of the scene appear further away, occupying fewer pixels in the rendered image. This is bad because it means we only have low-resolution imagery for the most important things in the scene! (These screenshots are also badly artifacted because for some reason the avi encoder we were using interlaces the output by default, and we couldn’t find a way to not interlace, without using a different codec! Hint to anyone making a video codec, anywhere in the past or future of this universe or any other: NOBODY LIKES INTERLACING. IT IS UGLY AND CAUSES A LOT OF PROBLEMS. DON’T DO IT. Please do your part to make the world a better place.)

It’s around this time that I realized we didn’t have to use a linear projection, which was good — if we can warp the image however we need to, then we can have the important parts of the scene landing really big in the middle of the texture, using lots of pixels, and squeeze the rest of the scene nonlinearly around the edges of the bitmap, using fewer pixels.

A 180-degree fisheye lens seemed like the right tool to do this. For other reasons that I won’t go into, another nice property of the fisheye lens (as opposed to some arbitrary distortion) is that it is physically plausible — you can mount a fisheye lens onto a physical camera and generate a similar image.

We found some plugins for Mental Ray (a renderer that you can use from Max or Maya) that seemed to do the trick. Here’s the output of one of the plugins we found (this is from Jan Sandström’s JS_fisheye.c, which appears to be a modification of a simple fisheye lens shader in the Mental Ray reference manual):

Unfortunately the field of view in this image we saved is less than 180 degrees, which makes it hard to directly compare with the later images, but just look at the basic character of it for now.

At first I assumed that this shader was implementing actual fisheye lens math, though there were no real comments to go on. Here’s the math used in the stripped-down Mental Ray Reference Manual:

    mi_vector_to_camera(state, &camdir, &state->dir);
    t = state->camera->focal / -camdir.z /
           (state->camera->aperture/2);
    x = t * camdir.x;
    y = t * camdir.y * state->camera->aspect;
    r = x * x + y * y;
    if (r < 1) {
        dir.x = camdir.x * r;
        dir.y = camdir.y * r;
        dir.z = -sqrt(1 - dir.x*dir.x - dir.y*dir.y);
        mi_vector_from_camera(state, &dir, &dir);
        return(mi_trace_eye(result, state, &state->org, &dir));
    }

(JS_fisheye.c has more parameters but is basically the same thing.)

Essentially, this code takes an input texture coordinate and converts it into a view vector in 3D space. In order to use the rendered image from within a shader in the game, I need to be able to invert this function: turn it from a view vector into a 2D texture coordinate. But when I tried to do this, I had all kinds of problems: the math was messy and ugly. I got the inkling that this shader may not be acting as a physical lens really would, but rather, was a 2D image-warping effect that gives the same general impression as a fisheye lens.

(The reason for thinking this is that light behaves bidirectionally; there’s no difference between light moving forward and light moving backward. So if something is simple when light is going in one direction, it ought to be simple in the other direction too. If the equation starts looking a lot more complicated, that is sort-of a violation of the way that the physical universe works, mathematically, and so one starts thinking that something is wrong. That was the intuition I had, anyway.)

Over on this page I found a nice reference for the way that a real fisheye lens bends light. Indeed the equation is very simple (though as it is written on that page, it’s not quite in the proper form for us to use). Based on that, I wrote a new Mental Ray shader that looks like this:

    mi_vector_to_camera(state, &camdir, &state->dir);
    miScalar x = state->raster_x / state->camera->x_resolution * 2 - 1;
    miScalar y = state->raster_y / state->camera->y_resolution * 2 - 1;
    
    miScalar r2 = x * x + y * y;

    if (r2 < 1) {
        miScalar c = 1 - r2;
        miScalar s = sqrtf(2 - r2);

        dir.x = x * s;
        dir.y = y * s;
        dir.z = -c;  

        mi_vector_from_camera(state, &dir, &dir);
        return mi_trace_eye(result, state, &state->org, &dir);
    }

Superficially it doesn’t look too different from the previous example, but in fact this version refracts light in a physical way and is invertible. Here’s what a render looks like using this lens:

As I mentioned, it’s hard to compare with the earlier shot because the field-of-view is different (sorry about that), but I think it’s evident that the nature of the distortion is fairly different between the two shots. In our version, it feels milder.

Here’s the runtime pixel shader code that inverts it:

    // (xprime, yprime, zprime) is the view vector in the same space where the environment map was rendered.

    float c = abs(zprime);

    video_uv.xy = float2(xprime, yprime);
    float scale = 1 / sqrt(1 + c);
    video_uv.xy *= scale;

    // Now we have video_uv in [-1, 1]; for [0, 1], scale appropriately.

It’s very simple, and the only real math required is a multiplication by a reciprocal square root (very fast for shaders!) So that was pleasant.

Once we had these both hooked up and working, the scene rendered perfectly, and we knew that the effect we wanted to create was achievable. By way of improving it, Ignacio suggested using a cylinder map instead of a fisheye lens, because that is better for this particular shot: we need to render a wide room, and see a lot laterally, but the vertical span is roughly constant and much shorter than the horizontal span. (The fisheye lens is more general, and we can use it for any scene, but to optimize texture resolution for specific cases, we might go to other things like the cylinder map. You can think of the cylinder map as being fisheyed along only one axis.)

Here’s the cylinder map version of the scene:

This is what we are going with for now. Problem solved, job seemingly well-done.

Ignacio put the cylinder mapping code into the same shader as the fisheye lens, and also added a latitude-longitude distortion.

Our work was sped up drastically by the fact that we were able to find JS_fisheye.c free on the internet, as well as information for the way a fisheye lens works. So here’s our attempt to give back a little: our final Mental Ray shader and the .mi file that defines the interface for it:

witness_fisheye.c
witness_fisheye.mi

If you are an experienced graphics programmer asking yourself why the hell didn’t they just use a cube map for this, well, there is a very good answer to that, but it involves the spoiler. You’ll see when the game is released!

The final result is published in Edge issue 223. I am very happy with it. It is probably my favorite interview I have done during my 15 years in the game industry. I thank Edge for the opportunity to do the interview and for their hard work in writing it up.

Because I like the interview so much, I should be happy when other news sites spread the word about it, right? Apparently it is not so. CVG just published online an article about the interview titled

They fill out their article by pasting in somewhat-out-of-context quotes to support this statement (and nothing else). To believe their account, it was a highly anti-Microsoft and pro-Sony interview. [I am not going to link to CVG's article, because I don't want to give them hits; go find it on a search engine if you want to see it.]

CVG’s article is a deceptive, manipulative piece of sensationalist crap meant to drive hits by stoking the argument between Sony fans and Microsoft fans. It misrepresents the content of the interview almost entirely. (Chris Hecker, who was half the interview, doesn’t even appear in their write-up. This alone should make it clear that they have no interest in truthful and accurate communication.)

Chris Hecker: Half the interview! He's even in the foreground in this picture.

For the record, here’s a list of the subjects Chris and I actually talked about:

• Overall trends in indie game development — where, and for how long, will indies be able to find Braid or World of Goo-sized audiences for their games?
• How it’s easier to make indie games, on the technical implementation side, than ever before.
• Small indie games versus big indie games — punk-rock-esque one-offs vs deeply exploring game design subjects.
• Indies ought to step up and produce higher-quality work rather than so many bad games and a few good ones.
• Braid is structurally a traditional game in many ways.
• The idea of game design as a stream of focused communication rather than a set of rules.
• Even the highest-selling games right now essentially sell to a niche audience.
• Innovation in mainstream games and how it helps drive the industry.
• When is an indie game really finished — how much work should you put in?
• Problems with the way video game education is set up right now.
• Most people who use the phrase “game mechanic” don’t seem to know what it means.
• How to see your game from the perspective of the player.

In the middle of all this, there is one bit about the way Microsoft and Sony, respectively, handle their portfolio strategies with regard to indie games. There are positives and negatives mentioned regarding each party. Sony does come out a bit ahead, but our discussion was much more even-handed than the confrontational CVG headline would have you believe.

From CVG's write-up, you would barely even know that I talked about my company's upcoming game The Witness.

Is it the case that I said the things that CVG quotes? Yes! Are they an honest representation of the content of the interview? No! They are a dishonest and manipulative way of driving hits to CVG’s web site.

Chris recently wrote about this kind of treatment at the hands of journalists, so rather than repeating him, I encourage you to go read that article. I do want to point out, though, that this is not a victimless crime.

When this kind of hack-job is written, it hurts indie developers: it harms our relationship with our business partners like Microsoft or Sony when they think that we called an interview just to trash them in public (or when they know we didn’t really do that, but they have to deal with the PR fallout of the public thinking we did, which is just as bad.) It harms my reputation with players because they think I called an interview just to make a big deal out of bashing Microsoft and generally being negative (which I did not do!) It does an injustice to people who want to read about thoughtful discussions in video games; it does this not only by failing to point out the thoughtful parts of the interview, but by injecting extra noise into the sphere of discussion, making it harder for everyone to find anything thoughtful.

But perhaps worst of all, CVG is crapping on their fellow journalists at Edge by depicting Edge’s hard work as tabloidesque junk.

Damage is clearly done, and CVG are doing this damage just so they can make a little bit of money off a few more web hits.

I encourage everyone out there to stop reading this kind of stuff. It’s just misinformation; it’s worse than worthless.

One interesting change, compared to the previous shots: you’ll notice that the lake that used to be in the center of the island, with the windmill on the edge, has moved over toward the left side and become a bay.

This is one of those interesting area where design and technical constraints interact heavily. We wanted the water of that lake to be reflective, but the way 3D graphics works, you basically need to render the entire scene an extra time in order to make a reflective surface. For outdoor scenes this is very expensive. Now, we also wanted some of the water out at the edge of the island to be reflective. If the inland lake and the water near the beach were at different elevations, this would mean we need to render the entire scene twice, all the time, to generate water reflections, before rendering the main scene. This can be optimized somewhat, but in an open-world game, there is only so much you can do.

The workaround to this was just to keep the inland lake at sea level, so that we can use a single rendering pass to generate all water reflections. Technically this worked fine, but design-wise, it meant that we had to have this big spot in the middle of the island that was at sea level, while also having everything at the edge of the island being at sea level (by definition). This gives the island a doughnut kind of shape, and given that the island is so small, this ended up severely restricting how many elevation changes can be introduced — the island was being artificially influenced to be flatter than it ought to be, because of this constraint of being sea-level in the middle.

So I moved that lake off to the side and connected it to the ocean. This involved moving a couple of other areas as well. The gameplay is preserved, and the new position is actually better for certain gameplay purposes. The island is still pretty flat, but I think you’ll see it getting less flat over time, now.

I had been viewing a bunch of online portfolios of 3D modelers (and lamenting the quality of these portfolios) with an eye to hiring someone. Sometimes these portfolios are shown in steps: here’s the wireframe, here’s the filled-in geometry, here it is with lighting, and here is the final thing after it’s been textured. Casey pointed out something interesting about these that I knew was true as soon as he said it: most of these portfolios look worse than they need to because the texturing ruins them. When you run a radiosity algorithm on a scene full of unpainted geometry, it has an elegant, sophisticated, scent-of-the-real kind of thing going on; but it is very easy to spoil this sophistication through texturing. And, it seems, in video games we have developed a tradition of “bad” texturing, whatever that means: unsubtle, unrefined, or just plain ugly.

We’re on a low budget here, being an indie game; so not only was it an open question how to solve the Ugly Texturing problem, but we also needed to solve it with relatively little manpower.

I knew that, ultimately, we could just fall back on untextured geometry with lighting, if we had good precomputed lighting. That’s why we spent so much time and effort putting together our lighting system (as has been written about many times in this blog). The question was, would we stick to vertex colors on most objects, or would we use some kind of texturing?

Early on, we identified a texturing style that would work for human-sized objects and would gel with the gameplay of The Witness. The style prioritizes color, and tries not to break up the geometry of the object by imposing conflicting details through the texture. We did some objects like a 1-person boat in this style in order to refine it. [I'd like to include a screenshot of the boat here to illustrate it, but the boat is messed up right now, so I'll just have to be vague.]

Because that style tries to use the texture map to accentuate the object, the contents of the texture must change with the local neighborhood of the object, which means you can’t use tiling textures very much. This means it would be very expensive to texture something like a big building, since you’d need unique texels everywhere.

Thankfully, the Western game development world has a tradition of developers being very open with their techniques and processes, and communicating with other developers very honestly about what is effective. (This is not something that one finds elsewhere in the world, and we should work hard to keep that spirit alive because it benefits everyone; but that’s a totally separate rant.) Here’s one good example of this kind of information exchange: at the GDC this year, some guys from Naughty Dog gave a talk on the art direction for Uncharted 2 (pdf link to their slides here.) Somewhere around page 220 of this very informative slideshow, they start talking about the blend shaders that they use in order to give more character to buildings and other geometry while still using tiling textures. The idea is that you interpolate between two (or more) textures with a blend function that fits the shape of your building nicely. You put some smarts into the interpolation so that it produces something that is visually interesting, rather than the blurry mess you would get via naive interpolation.

For The Witness it seemed like a good idea to use this kind of technique, but in a minimalist way, to create patterns that accentuate the shapes of buildings, and help tie them together visually, without stumbling into the realm of Ugly Texturing.

(There are a lot of reasons for the minimalism; one reason, though, is to provide clarity about what you can interact with, versus what is inert. Interactive things are colored in a way that makes them stand out in an obvious way. Think of it as something like what Mirror’s Edge did, but in a different way.)

We’ve started experimenting with these blend shaders; here are some images:

The upper-left shows a scene before we implemented the blend shader for buildings. In the upper-right, we are using a blend shader that blends the original texture with an alternate one that is a bit darker and rougher (but the difference is a bit subtle, by game standards at least). Below these images are the same scenes with precomputed lighting mixed in.

It is pretty clear to me that the lighting is actually the biggest ingredient in tying the scene together, but the blend shader is, I think, a good extra kick. It helps the area feel like there is a little bit of life and history to it (for example, the lighter-colored paths near the puzzle panels in the foreground, which imply that people have been walking that way). What you see here is not of shipping quality, but I see how we can reach shipping quality by iterating on this style and putting production-level effort into the assets.

Here’s a close-up of what the stairway looks like:

As you can see, the texture maps have some grain to them; from far away, the grain melts away and the structure is mostly a solid color, accented by lighting.

These experiments are still early yet but I find them very promising. The plan is that we will continue to hone the style and end up with something very special.

This talk is called Games and the Human Condition; it was given at Rice University on September 27, 2010. It’s about “best practices” of modern game design that I find unsettling, and the way in which “social games” and “gamification” are destructive.

A full video is available from this page at Rice University.

Here’s a link containing just the slides (ppt format).

Here’s a long and wordy abstract:

Video games have evolved tremendously over the past few decades; they’re much more entertaining than they used to be. That is not by accident; we, the community of game designers, have been continuously refining our techniques. The most common way we do this is by testing out our games on you, the players, and optimizing for the “best” result (where “best” is defined by us). As this process is ongoing, what kind of relationship exists between the designer and the player? Is it artist/audience, experimenter/subject, entrepreneur/customer, or tycoon/resource? Invariably it’s some admixture of these things, the particular ratios for a given game being chosen by its designers (usually without awareness that a decision is being made). Today, due to the way the Internet is widely used, and because game designers are becoming more serious about certain aspects of their craft, the iteration time of this game design optimization process is shorter than ever before: designers can observe their players much more thoroughly, and more quickly, than they ever have in the past. At some point a quantitative change becomes a qualitative one: the result of all this competency may be heavily destructive. Some aspects of the current notion of “good game design” may in fact be very bad, or at least indefensible, from an ethical standpoint. Today’s “better” video games spend a great deal of effort to undermine defenses that took you tens of millennia to evolve. They tend to be successful at this. As designers keep evolving their craft and gain greater analytical power, what will happen?

I’ve seen mixed reactions to these mazes and to the gameplay shown in the video. Some people seem to have jumped to the conclusion that the game is going to be about solving simple maze puzzles, and thus not be particularly exciting.

If that thought crosses your mind, all I can ask is to have faith. I wouldn’t make a game about solving a series of rote puzzles. The footage came from the beginning of the game because we don’t want to spoil anything major. But even this beginning-of-the-game stuff has more to it than meets the eye.

It’s like Braid, in a way. Back before Braid was released, I never made a trailer video for it, because I couldn’t come up with a way to show in video form what the game was about. You can show some rewinding happening, or other funky time stuff, sure, but what is cool about Braid is not the mechanics of those things or how they look on the screen. What is cool about Braid is what happens in the player’s mind during the puzzle-solving process. The time stuff is the method by which that is implemented, but the rules of time manipulation do not contain the magic. Watching a video, it’s nearly impossible to see the magic. You have to play the game to see the magic.

The Witness is like that too. The blue mazes are a major aspect of the game, but they are not the point. The point is the magic that happens in the player’s mind when he understands the subtle things that the mazes are saying — because the mazes aren’t just puzzles, they are lines of communication that aggregate, become more complex and eventually say surprising things. This can’t be seen in a video; you have to play it to have the experience. The magic is in the player’s head, not on the screen.

The only two guys who have played The Witness to completion have said it will be better than Braid when it is done. I am certainly not going to jump up and down and say “hey this game is better than Braid”, or even claim that Braid is good. But I just want to put that out there as reassurance to those of you who are worried about that gameplay video.

I have a file with about 80 game ideas in it, most of which would probably make really good games. But I can only make one game at a time right now, so I had to pick one. I picked The Witness, even though it more difficult and expensive to make than most of the other ideas, because I was more excited about the core concept of The Witness than about the other 79 ideas. This is a highly ambitious game that stretches my ability as a game designer to its limit.

So if you don’t yet see why the game is ambitious or exciting, please have patience! We’ll be releasing more information as the game comes closer to completion…

And whatever you think about what you’ve seen so far, I’d like to thank you for your interest in the game!

Using the same engine makes it very easy to keep the global illumination in sync with the rest of the game. If we add new shaders or materials, change the representation of the geometry or alter the world, keeping the global illumination up to date requires little o no effort. Another interesting benefit is that optimizations to the rendering engine also benefit the global illumination pre-computation. However, as we will see later, in practice each application stresses the system in different ways and end up limited by different bottlenecks, what improves the rendering time does not necessary improve the pre-computation and vice-versa.

Projection Methods

I tried several different approaches to render the scene and estimate the incoming radiance. The most common approach is to render hemicubes; that requires rendering the scene to 5 different viewports. Several sources suggested the use of spherical or parabolic projections, but in my experience these approaches produced terrible results due to the non linear projection. Without very high tessellation rates the scene would become unacceptably distorted, creating gaps between objects or making parallel surfaces self-intersect.

In GPU Gems 2, Coombe et al. suggested to project the z value independently of (x,y) coordinates using an orthographic projection in order to maintain a correct depth ordering. As seen in the next picture, that removed the most obvious artifacts; note how the staircase inside the tower is not visible anymore:

However, while this works OK from some points of view, the distortion becomes unacceptable when the camera is close to a surface, which is always our case. So, in practice these approaches are not very useful. Note how the outdoors are visible when sampling the lighting from the interior of the tower:

Other implementations use a single plane projection. This kind of projection does not capture the whole hemisphere, but by using a wide field of view it’s possible to capture about 80% of it. I haven’t actually tried this approach, but to get an idea of what the results would look like, I modified the hemicube integrator to ignore 20% of the samples near the horizon and found out that this introduced artifacts that were unacceptable for high-quality results.

Pascal Gautron estimates that on typical scenes this approach introduced a 18% RMS error, but suggests various tricks to reduce it by re-balancing the solid angle weights, or extrapolating the border samples and filling the holes. The latter method reduces the error down to a mere 6%.

However, these improvements do not eliminate banding artifacts caused when an intense light source visible from one location is suddenly not visible anymore in the next when it falls out of the projection frustum. To make things worse, the projection lacks rotational invariance, which enhances the problem when hemicubes are rendered with random rotations. So, while in many cases the results look alright, the few cases that do not render the method impractical.

Another problem is that the wide frustum is 4 times larger than the equivalent hemicube and easily intersects with the surrounding geometry; as I’ll discuss next, this is one of the main problems that I had to deal with.

In the end I simply stayed with the traditional hemicubes for simplicity. I think it would have been possible to do somewhat better using a fancy tetrahedral or pyramidal projection with 3 or 4 planes, but that didn’t seem worth the trouble.

Hemicube Overlaps

As I just mentioned, one of the main problems that I had to solve was to handle overlaps between the hemicubes and the geometry. These overlaps are unavoidable unless you impose some severe modeling restrictions that are not desirable for production.

When the geometry is well formed and does not have self-intersections, these overlaps can sometimes be avoided with centroid sampling. Without centroid sampling the location where the hemicube is rendered from can easily end up being outside of the polygon that contains the sample and thus inside another volume.

With centroid sampling the origin of the hemicube is now always inside the surface. However, it can still be very close to the surface boundary and have overlaps with nearby surfaces. A simple solution is to move the near plane closer to the camera until the overlap dissapears, but how close? The distance to the nearest boundary can be determined analytically, but there’s a limit in how much the near plane distance can be reduced without introducing depth sampling artifacts. An interesting solution for this problem is to render multiple hemicubes at the same location, each hemicube enclosing the previous one with its far plane at the near plane of the next.

That would solve most the problems when the geometry is watertight and centroid sampling is used, but in many cases our assets are not always well conditioned. So, I had to find a more general solution.

I tried all sort of heuristics, to render backfaces in black, to fill them with the average color, to extrapolate the nearest colors, etc. None of these solutions worked well in all cases. In the end, something that produced satisfactory results was to render the backfaces with a special tag to distinguish them from front facing faces when integrating the hemicubes. Whenever a hemicube had too many tagged texels it would be discarded entirely, and instead its output radiance would be estimated at that location interpolating or extrapolating it from the nearest valid samples.

The extrapolation is not always correct, and the detection of invalid hemicubes is not 100% accurate, but so far this is the method that I’m happiest with. In the next article I’ll explain how this extrapolation is done in more detail.

Shadow Rendering

While the goal was to use the same rendering engine, using its shadow system ‘as is’ would have been overkill. We are using cascade shadow maps, which are updated on every frame, and to maximize the shadow map resolution the shadow maps are tightly aligned to the view frustum slices. When rendering the hemicubes, shadow map resolution is not that important, and re-rendering the shadows for each hemicube face is unnecessarily expensive. To keep things simple I still used cascade shadow maps, but instead of rendering them for each view, I only rendered them once for each object positioning the cascades around the center of the object.

Rasterization

Now that I’ve explained how to capture the irradiance of the scene at any point, it’s necessary to sample it for every lightmap texel. To do that I simply rasterize the geometry in the lightmap UV space using a conservative rasterizer and render one hemicube at every fragment whose coverage is above certain threshold.

One nice thing of using our own rasterizer is that I can evaluate the exact coverage analytically. Since the parameterization does not have overlaps, the coverage is just the area of the intersection of the texel square with the polygon being rasterized. Similarly, the centroid is the center of mass of the clipped polygon.

This approach gives us a position and a direction to render the hemicube from, but I still need to determine the orientation of the hemicube. Initially, I simply chose an arbitrary orientation using the traditional/standard method, but that resulted in banding artifacts:

The source of the problem is just the limited hemicube resolution, but it’s surprisingly hard to get rid of it entirely by only increasing the hemicube size. A better solution is to simply use random orientations:

Note that I am trading banding artifacts by noise, but the latter is much easier to get rid of by using some smoothing.

Integration

Once the hemicubes have been rendered it’s necessary to integrate the sampled irradiance to compute the output radiance. The integral of the irradiance hemicube is simply a weighted sum of the samples, where the weights are the cosine-weighted areas of the hemicube texels projected on the hemisphere, that is, the cosine-weighted solid angles of the texels. These weights are constant, so they can be precomputed in advance.

In the article by Hugo Helias that I referenced in the introduction the same approach is used and this weight table is called the “The Multiplier Map”. Beware of that part of the article, though, because there’s a bug in the formulas. The subtended solid angle of the texel is not “the cosine of the angle between the direction the camera is facing in, and the line from the camera to the pixel”:

That term needs to be divided by the squared distance between the location of the camera x and the center of the texel y:

See formula 25 in the Global Illumination Compendium for more details.

Also, since this is a precomputation it made sense to evaluate the solid angle exactly. That can be derived using Girard’s formula or directly using the handy formula in Manne Ohrstrom’s thesis. Doing so improved the invariance of the integral with respect to rotations of the hemicube and that reduced noise significantly.

In the end, the integration is a fairly simple operation. It could be done very efficiently using compute shaders and I might still do that in the future, but for prototyping purposes it was much easier to perform the integration in the CPU. Unfortunately, that required transferring the data across the bus and, if not done properly, it severely reduced performance.

Asynchronous Memory Transfers

Contrary to popular belief, Direct3D 9 provides a mechanism to perform memory copies asynchronously. In Direct3D 10 and OpenGL this is done through staging resources or pixel buffer objects; in Direct3D 9 something similar can be achieved with offscreen plain surfaces. The problem is that this is not very well documented, it’s very easy to fall out of the fast path, and the conditions in which that happens are slightly different for each IHV.

In the first place, you have to create two render targets and offscreen plain surfaces:

dev->CreateRenderTarget(..., &render_target_0);
dev->CreateRenderTarget(..., &render_target_1);
 
dev->CreateOffscreenPlainSurface(..., &offscreen_surface_0);
dev->CreateOffscreenPlainSurface(..., &offscreen_surface_1);

If you have more than two targets the copies become synchronous in some systems, but two is sufficient to generate enough work to do in parallel with the memory transfers.

The most important observation is that the only device method that appears to be asynchronous is GetRenderTargetData. After calling this method, any other device method blocks until the copy is complete. So, instead of overlapping the memory transfer with the rendering code, you have to find some CPU work to do simultaneously. In my case, I just do the integration of one of the render targets while copying the other:

while hemicubes left {
 
    [...] // Render hemicube to render_target_0
 
    // Initiate asynchronous copy to system memory.
    dev->GetRenderTargetData(render_target_0, offscreen_surface_0);
 
    // Lock offscreen_surface_1
    // at this point the corresponding async copy has finished already.
    offscreen_surface_1->LockRect(..., D3DLOCK_READONLY);
 
    [...] // Integrate hemicube.
 
    offscreen_surface_1->UnlockRect();
 
    swap(render_target_0, render_target_1);
    swap(offscreen_surface_0, offscreen_surface_1);
}

Another problem is that the buffer needs to be sufficiently big to reach transfer rates that saturate the available bandwidth. In order to achieve that I batch multiple hemicubes in a single texture atlas as seen in the following picture:

In practice I do not render the hemicubes using the depicted cross layout, but instead arrange the faces tightly in a 3×1 rectangle that does not waste any texture space and maximizes bandwidth. However, the cross layout was useful to visualize the output for debugging purposes.

Performance and Conclusions

The performance of our global illumination solution is somewhat disappointing. While we do most of the work on the GPU we are not really making a very effective use of it. The main GPU bottleneck is in the fixed function geometry pipeline; we are basically rendering a large number of tiny triangles on a small render target.

But that’s not the main problem. Traversing the scene in the CPU and submitting commands to render it actually takes more time than it takes the GPU to process the commands and render the scene, so the GPU is actually underutilized.

Our engine was designed for quick prototyping, to make it very easy to draw stuff with different materials, but not to do so in the most efficient way possible. These inefficiencies became a lot more pronounced in the lightmap baker.

On the bright side, as we optimize the CPU side of the engine we expect performance to improve. Modern GPUs also have functionality that make it possible to render to multiple hemicube faces at once. We expect that will also reduce the CPU work significantly.

While it was easy to get our global illumination solution up and running by reusing our rendering engine, bringing it to a level of sufficient quality and performance took a considerable amount of time. In the next installment I’ll describe some of the techniques and optimizations that I implemented to fix some of the remaining artifacts and improve performance.

Here’s an example. The following tree looked OK close to the camera:

but as we moved away, the leafs started to fade and thin out:

until they almost disappeared:

I had never encountered this problem before, neither had I heard much about it, but after a few google searches I found out that artists are often frustrated by it and usually work around the issue using various hacks. These are some of the proposed solutions that are usually suggested:

• Manually adjusting contrast and sharpening the alpha channel of each mipmap in Photoshop.
• Scaling the alpha in the shader based on distance or on an lod factor estimated using texture gradients.
• Limiting the number of mipmaps, so that the lowest ones aren’t used by the game.

These solutions may work around the problem in one way or another, but none of them is entirely correct and in some cases add a significant overhead.

In order to address the problem it’s important to understand why the geometry fades out in the distance. That is because when computing alpha mipmaps using the standard algorithms, each mipmap has a different alpha test coverage. That is, the proportion of pixels that pass the alpha test changes, in most cases going down and causing the texture to become more transparent.

A simple solution to the problem is to find a scale factor that preserves the original alpha test coverage as best as possible. We define the coverage of the first mipmap as follows:

coverage = Sum(a_i > A_r) / N

where A_r is the alpha test value used in your application, a_i are the alpha values of the mipmap, and N is the number of texels in the mipmap. Then, for the following mipmaps you want to find a scale factor that causes the resulting coverage to stay the same:

Sum(scale * a_i > A_r) / N == coverage

However, finding this scale directly is tricky because it’s a discrete problem, in general, there’s no exact solution, and the range of scale is unbounded. Instead, what you can do is to find a new alpha reference value a_r that produces the desired coverage:

Sum(a_i > a_r) / N = coverage

This is much easier to solve, because a_r is bounded between 0 and 1. So, it’s possible to use a simple bisection search to find the best solution. Once you know a_r the scale is simply:

scale = A_r / a_r

An implementation of this algorithm is publicly available in NVTT. The relevant code can be found in the following methods of the FloatImage class:

float FloatImage::alphaTestCoverage(float alphaRef, int alphaChannel) const;
void FloatImage::scaleAlphaToCoverage(float desiredCoverage, float alphaRef, int alphaChannel);


And here’s a simple example of how this feature can be used through NVTT’s public API:

// Output first mipmap.
context.compress(image, compressionOptions, outputOptions);
 
// Estimate original coverage.
const float coverage = image.alphaTestCoverage(A_r);
 
// Build mipmaps and scale alpha to preserve original coverage.
while (image.buildNextMipmap(nvtt::MipmapFilter_Kaiser))
{
    image.scaleAlphaToCoverage(coverage, A_r);
    context.compress(tmpImage, compressionOptions, outputOptions);
}

As seen in the following screenshot, this solves the problem entirely:

even when the trees are far away from the camera:

Note that this problem does not only show up when using alpha testing, but also when using alpha to coverage (as in these screenshots) or alpha blending in general. In those cases you don’t have a specific alpha reference value, but this algorithm still works fine if you choose a value that is close to 1, since essentially what you want is to preserve the percentage of texels that are nearly opaque.

For all three days of PAX 2010, The Witness was publicly playable by anyone who came by the booth.  However, it was unmarked and unattended, so it was easy to miss (as many people did; but then, plenty of people played it, too).

Here’s a close-up, with the opening scene of the game, as it silently waits for someone to sit down and play:

I had several reasons for wanting to show the game this way.  Firstly: At a show full of companies trying to capture your attention and sell you things, I wanted to do something that is subtle, and a surprise — if you notice it, and decide to investigate, you find something unexpected.

Also, I wanted people to be able to play the game for as long as they want, not feeling pressured to stop playing because of a huge line of antsy people waiting behind them.

Before PAX, only two people had ever played the game; those people were game developers, and the versions they played were very rudimentary.  This was the first time the game had ever been played by general players, and I wanted to see how people would react to it, as honestly as possible.  I didn’t want to give them any reason to think that this was a good game, or was by any designer they might have heard of.  So I just left the game running, and would occasionally peek at it from afar to see who was playing and how they were doing.

PAX is a huge show, with tons of stuff to see everywhere; if the game could hold peoples’ attention under those conditions, purely on its own merits rather than by hype or pedigree, then I would know that it was really working.

Happily, the game worked very well — despite the fact that the visuals are still in an early prototype stage.  People tended to play for a long time, and they liked what they played.  Here’s a picture from Sunday; these guys played for a long time, perhaps an hour and a half:

Some game designers did recognize the game; here’s Dylan Fitterer (of AudioSurf) playing:

Stephen Totilo of Kotaku recognized the game early, on Friday, and found me camping out on the other side of the booth.  Thankfully, though, he didn’t spill the beans.

I am very happy with how this playtest session went.  I am glad I was able to bring a game directly to players, as a surprise.

The common wisdom among game designers is that the first time you put your game in the hands of players, it totally falls apart, and nothing works the way you expected.  Thankfully, that didn’t happen here.  The game does work as expected, despite being kind of crazy and experimental, and people enjoy playing it.

All that’s left now is just to do all the hard work required to finish the game!

Update: Stephen Totilo has posted his impressions, along with a gameplay video, at Kotaku.

A few of the many, many ideas we looked at:

Here’s David, looking like he’s at the top of his game:

We’ll be working on that ugly green area soon…

Today I took a step in that direction by moving some areas around and compacting the island, changing its shape a bit:

Here’s a shot from the other side of the island, where you can see that there is plenty of empty space left:

So-me of this will be filled with new gameplay, some will be removed or reshaped, etc, depending on how things evolve.

Here’s a close-up of a new area; there’s something going on there with a lot of wires, and stuff:

A movie can give you a satisfying experience in 2 hours.

A painting or a sculpture can give you a satisfying experience in 10 minutes.

A song can give you a satisfying experience in 3 minutes.

What is it about certain kinds of linear-experience games that makes players feel they need to play them for hours upon hours in order to accrue a satisfying experience? If it really takes that long, doesn’t that mean the medium is somehow deficient in important ways?

Gamers seem to praise games for being addicting, but doesn’t that feel a bit like Stockholm syndrome? If you spend 20 hours playing a game, but the good parts could have been condensed into 3, then didn’t you just waste 17 hours?

If you waste 17 hours a month for the rest of your life, what is the cost of that, socially, quality-of-life-wise, economically, or however else you want to measure?

I am not saying that all games should be short. Certainly there’s a place for long-form games, just as there’s a place for long-form TV shows like Deadwood or Lost. But if we lived in a world where all cinema had to be the length of Lost, and films like Inception were criticized by cinema fans, and given lower scores in reviews and such, for being too short… wouldn’t that be perverse? Certainly the people in that world would be ignoring a big chunk of the potential of that medium, for arbitrary and weird reasons.

This post is part of an industry-wide commentary by indie developers on the subject of short games. Other participants:

Ron Carmel of 2DBoy

Chris DeLeon

Dave Gilbert of Wadjet Eye Games

Matt Gilgenbach of 24 Caret Games

Michael Todd
Eitan Glinert of Fire Hose Games

Cliff Harris of Positech Games

Chris Hecker of Spy Party

Scott Macmillan of Macguffin Games

Noel Llopis

Peter Jones of Retro Affect

Lau Korsgaard

Martin Pichlmair of Broken Rules

Greg Wohlwend of Intution Games

Jeffrey Rosen of Wolfire

Steve Swink

The Chaparral High School Alumni Theatre’s production of Braid: A Movement Piece during the summer of 2010. Using a unique combination of content movement and contact improvisation dance techniques, Braid follows the story of young scientist Tim on his quest to find a Princess.

We are starting to tinker around with the sky (this one is way too dark). Also, moving around a bunch of locations and adding some new ones.

I’m doing a couple of weeks’ worth of serious map organization / reorganization in order to get the game to a point where we can playtest it at pretty much any time, and whoever sits down to play will have a reasonable experience. The game was in this kind of shape back in January, but we had to do a lot of work on game internals in order to get things where they needed to be, interactivity-wise, for the mid-stage of development. Now that most of that stuff is done, it’s time to put the world/content back in playable shape; from there, new stuff can be added in a way that continuously fills out the world. Once I feel like the world is “full enough” we can leave the preproduction phase and start going into production (which is where we start caring about what the graphics look like, start seriously doing voice recordings, etc).

One area I’ve been working on for the past few days is the start of the game. It is meant to be a semi-guided experience that clues the player into what the game is about. (If you remember the beginning of Braid, it’s something like that.) You may have seen the interior of the house (the white building on the left) in earlier blog posts:

The area just outside the house is a constrained path, but very soon the player exits this area and can freely roam an open world.

Most open-world games want to impress you with the hugeness of the world, and the corollary to that is that you spend a lot of time walking / riding / driving between places. The Witness takes the opposite philosophy: it wants the world to be as high-density as possible, just packed full of interesting things. As the island develops, we will be placing areas as close to each other as we can, within constraints. (Braid had a similar high-density philosophy — though it manifested in a different way — and many people appreciated that).