In this instalment, Thiel takes the question of what kind of company to start and reframes it in terms of secrets.

Thiel’s point of view reminds me of the way I think about games in general. Quite often I go out and give speeches about game design or business, and I say things that some think are overly idealistic, or oblivious to the necessities of recipients’ situations, or just plain wrong. Certainly my advice usually runs counter to conventional wisdom. (Yet somehow, by following these principles, I seem to do okay.)

Thiel begins with the question: “What important truth do very few people agree with you on?” and goes on to define secrets, in this context, as “unpopular or unconventional truths”. If everyone knew these things and believed them, they wouldn’t be secrets.

I recommend this write-up to anyone who wants to think unconventionally about game design.

The changes are a little bit drastic in some spots, but overall not as drastic as I had expected. I had been worried about some issues in the natural flow of where players would naturally end up going, especially early in the game. It turns out we were able to solve these without changing too much. So between last island update and this one, for the most part, some areas were moved laterally or expanded, and some new elevation changes were introduced, but areas were not shuffled around; they are all more-or-less in the same place with relation to each other. So for the most part, things are converging, which is a good sign.

And, oh yeah, we are making that mountain a lot bigger. It is the messiest part of the island right now, and has been for a while. There are still a few things gameplay-wise to figure out here, then it will start to come together. My biggest questions in the area have to do with that white building near the bottom of the image; the puzzles inside that building will determine its overall structure, which will determine how it is worked into the surrounding terrain of the mountain. We have some proposals from the architects on what this building will look like, but it is waiting on me to finish the puzzle design.

Because we haven’t done it in a while, here’s the island from a different angle:

The current puzzle count for the whole game is 450.

Just before I left my last job I was working as a Lead/Art Director on a marketing driven project focused on selling as many units as possible. On one side I had high level decisions being made by stockholders whose focus is on turning a profit while on the other side there was the development team who truly believed they would be able to make a good game.

Also I was surrounded by schedules, performance reviews, hierarchical titles, producers, human resources, bonus system and all the other corporate systems that eventually dilute the organic process of doing something creative. While I understand the logic behind it I  realized that is not the reason why I joined the video games industry and that something wasn’t right.

Now, after these few months, the answers for these questions became clear. Its all based on trust. In this project, all the decisions we arrive at are very organic. Jonathan has a very clear vision of what he wants and the fact that we can see it and believe in it makes it extremely logical and simple.

Our passion for what we do (and what we would probably be doing on our free time if we weren’t being paid for it) gets distilled directly into the game in a very rewarding process.How much and how we give it depends entirely on us and that creates a natural challenge and pressure that most AAA companies cannot provide.

This trust leads to respect and it creates a very unique work environment. This is something that I’ve never seen with all the project management, task scheduling and scrum meetings that try to fix an inherently broken game. I don’t think video games can be produced like an assembly line, at least not games that have something meaningful to say, and I’m glad that I found people that are doing it the right way.

All this may sound trivial but probably 90% of the game developers that I know will understand how big of an issue this is. I still believe big companies can change their ways and make the same profit if not more, but it has to start from the bottom, the people who are hard at work making the games.

As for the second part, I would like to share what we have been trying to do with the many island panels. As you might already know the player has to solve different puzzles using these panel interfaces scattered around the island.

I started my research by looking at what materials they could be made of, what technology they would use and how they would be assembled and how all this could relate with the different areas and narrative of the game.

This first set of images shows some proposals of how the panel screens could possibly look. I tried not to focus so much on the gameplay but more on ideas that could lead to further discussions among the team.

After some solid feedback and a couple of art meetings I was able to understand what was working or not and decided to do a new set of proposals. Also this time I tried to focus on the more technical aspect of getting these panels to work in the game and if the effort we would have to put into them would be worthwhile.

The same amount of work went into the structure of the panels and how they would be assembled. This time, working with the architects gave valuable input into understanding how they would actually be built in the environment and the amount of work required. On this image you can see a screen capture of several prototypes before being tested ingame.

These are usually modelled very fast with just base colors so I can do several iterations until I find something that I’m happy with.
And finally here is how some of them are looking in-game. They are still in concept but by placing them in the correct context makes it a lot easier to understand what is working or not.

But today we are going to have the first in a series of meetings wherein we think pretty hard about the topography of the island and what the flow of natural curious investigation will tend to be from various spots. Expect the island to look pretty different in the next update! (Or maybe it will turn out that things are mostly fine…)

Lately, I have been working on some of the more sophisticated puzzles in the game. These are things that were at least sort-of done already, but I wanted to revise the designs to improve them — to ensure that the puzzles are as interesting as possible and built as closely as possible around the core themes of the areas that contain them. I am happy with the way these are coming along so far, but there is definitely still some work to do here.

The art team has been working further to nail down the style of the game, now focusing on cliffs and plants and other natural things, as well as control panels and other bits of machinery. We’re also taking a few buildings that the architects have finished designing, and starting to build out the geometry as one really would for a video game. (The architects give us buildings modeled in SketchUp, and they tend to come into the game a bit messy, but also, not modeled the way an experienced video game modeler would do it).

On the programming side, we’ve got lots done lately. Andy has added a software occlusion culling system, which should help with frame rates in the game. He’s also been playing with culling for water reflections. Salvador got animations working in the exporter and in the game, so now we can play animations on entities, which is very useful. (We worked on the game for years and built something fully playable without this ability!) I have been going back and doing the gameplay work for animated entities, as some things have to change in the gameplay logic. In all it is a very nice improvement. Salvador is now working on asset streaming, which will help the game start up faster and run better (and help it run at all on lower-end platforms like the iPad). Ignacio’s been doing a whole lot of fixes to various systems and hammering away at the todo list. One of these things has been better control for the tone mapping process. We now get a graph of tone mapping curve:

And we also get the ability to export screenshots into Photoshop that have a color key on them; then the colors can be tweaked in Photoshop and re-imported into the game to alter tone mapper settings. This is in many cases more convenient than playing with those sliders.

We had to do a lot of work to get the game back into playable condition; a lot of things had been broken since the press tour last August / September. (When you are making sweeping changes to a big and complicated game world, it is pretty easy for things to get bent out of shape). So the GDC served as a good internal deadline to have the game banged back into shape.

I didn’t expect a lot of press coverage to come out of this, because at the GDC there’s just a lot of stuff going on, a lot of things for people to pay attention to and write about. So I had been doing this mostly as an opportunity to keep the press up-to-date with the game, see how it is evolving graphically, etc. But, to my surprise, we got a lot of coverage. Here are links to what people are saying about the game lately:

First, a video interview on Gamespot (excerpted; we probably talked for 45 minutes or an hour).

Kirk Hamilton at Kotaku finds a musical treatment for The Witness’ game design philosophy.

Ben Kuchera at the Penny Arcade Report, with a piece about online distribution channels, and another biz-focused one.

Ben Gilbert at Joystiq, with a design-oriented posting and also a more biz-like one.

Alex Rubens from G4 with an overall preview of the game.

Daniel Starkey of Destructoid, with a preview as well.

As you can see from the write-ups, people seemed to really dig the game. So that is all good.

Keep in mind that, because I am known for my previous game and and the reporters are there talking directly to me, the tone that tends to get adopted in these write-ups makes it sound as though I were making the game myself. That’s not the case; there’s a good-sized team of very talented people building the game! (This reminds me that we need to update the About page to list everyone who is currently working on the project.)

When textures are static a common solution is to pre-process them in an attempt to eliminate the edge seams. In a short siggraph sketch, John Isidoro proposed averaging cube map edge texels across edges and obscuring the effect of the averaging by adjusting the intensity of the nearby texels using various methods. These methods are implemented in AMD’s CubeMapGen, whose source code is now publicly available online. While this seems like a good idea, a few minutes experimenting with CubeMapGen make it obvious that it does not always work very well!

Embedded Texture Borders

A very simple solution that even works for dynamic cube maps is to slightly increase the FOV of the perspective projection so that the edges of adjacent faces match up exactly. Ysaneya shows that in order to achieve that, the FOV needs to be tweaked as follows:

fov = 2.0 * atan(n / (n - 0.5))

where n is the resolution of the cube map.

What this is essentially doing is to scale down the face images by one texel and padding them with a border of texels that is shared between adjacent faces. Since the texels at the face edges are now identical the seams are gone.

In practice this is much trickier than it sounds. While the fragments at the adjacent face borders should sample the scene in the same direction, rasterization rules do not guarantee that in both cases the rasterized fragments will match.

However, if we take this idea to the realm of offline cube map generation, we can easily guarantee exact results. Cube maps are often used to store directional functions. Each texel has an associated uv coordinate within the cube map face, from which we derive a direction vector that is then used to sample our directional function. Examples of such functions include expensive BRDFs that we would like to precompute, or an environment map sampled using angular extent filtering.

Usually these uv coordinates are computed so that the resulting direction vectors point to the texel centers. For an integer texel coordinate x in the [0,n-1] range we map it to a floating point coordinate u in the [-1, 1] range as follows:

map_1(x) = (x + 0.5) * 2 / n - 1

We then obtain the corresponding direction vector as follows:

dir = normalize(faceVector + faceU * map_1(x) + faceV * map_1(y)

When doing that, the texels at the borders do not map to -1 and 1 exactly, but to:

map(0) = -1 + 1/n
map(n-1) = 1 - 1/n

In our case we want the edges of each face to match up exactly to they result in the same direction vectors. That can be achieved with a function like this:

map_2(x) = 2 * x / (n - 1) - 1

If we use this map to sample our directional function, the resulting cube map is seamless, but the face images are scaled down uniformly. In the first case the slope of the map is:

map_1'(x) = 2 / n

but in the second case it is slightly different:

map_2'(x) = 2 / (n - 1)

This technique works very well at high resolutions. When n is sufficiently high, the change in slope between map_1 and map_2 becomes minimal. However, at low resolutions the stretching on the interior of the face can become noticeable.

A better solution is to stretch the image only in the proximity of the edges. That can be achieved warping the uv face coordinates with a cubic polynomial of this form:

warp3(x) = ax^3 + x

We can compose this function with our original mapping. The result around the origin is close to a linear identity, but we can adjust a to stretch the function closer to the face edges. In our case we want the values at 1-1/n to produce 1 instead, so we can easily determine the value of a by solving:

warp3(1-1/n) = ax^3 + x = 1

which gives us:

a = n^2 / (n-1)^3

I implemented the linear stretch and cubic warping methods in NVTT and they often produce better results than the methods available in AMD’s CubeMapGen. However, I was not entirely satisfied. While this removed the zero-order discontinuity, it introduced a first-order discontinuity that in some cases was even more noticeable than the artifacts it was intended to remove.

You can hover the cursor over the following image to show how the warp edge fixup method eliminates the discontinuities, but sometimes still results in visible artifacts:

Any edge fixup method is going to force the slope of the color gradient across the edge to be zero, because it needs to duplicate the border texels. The eye seems to be very sensible to this form of discontinuity and it’s questionable whether this is better than the original artifact. Maybe other warp functions would make the discontinuity less obvious, or maybe it could be smoothed like Isidoro’s method do. At the time I implemented this I thought the remaining artifacts did not deserve more attention and moved on to other tasks.

Modifed Texture Lookup

However, a few days ago Sebastien Lagarde integrated these methods in AMD’s CubeMapGen. See this post for more results and comparisons against other methods. That got me thinking again about this and then I realized that the only thing that needs to be done to avoid the seams is to modify the texture coordinates at runtime the same way we modify them during the offline cube map evaluation. At first I thought that would be impractical, because it would require projecting the texture coordinates onto the cube map faces, but turns out that the resulting math is very simple. In the case of the uniform stretch that I first suggested, the transform required at runtime is just a conditional per-component multiplication:

float3 fix_cube_lookup(float3 v) {
   float M = max(max(abs(v.x), abs(v.y)), abs(v.z));
   float scale = (cube_size - 1) / cube_size;
   if (abs(v.x) != M) v.x *= scale;
   if (abs(v.y) != M) v.y *= scale;
   if (abs(v.z) != M) v.z *= scale;
   return v;
}

One problem is that we need to know the size of the cube map face in advance, but every mipmap has a different size and we may not know what mipmap is going to be sampled in advance. So, this method only works when explicit LOD is used.

Another issue is that with trilinear filtering enabled, the hardware samples from two contiguous mipmap levels. Ideally we would have to use a different scale factor for each mipmap level. That could be achieved sampling them separately and combining the result manually, but in practice, using the same scale for both levels seems to produce fairly good results. We can easily find a scale factor that works well for fractional LODs as a function of the LOD value and the size of the top level mipmap:

float scale = 1 - exp2(lod) / cube_size;
if (abs(v.x) != M) v.x *= scale;
if (abs(v.y) != M) v.y *= scale;
if (abs(v.z) != M) v.z *= scale;

If you are using cube maps to store prefiltered environment maps, chances are you are computing the cube map LOD from the specular power using log2(specular_power). If that’s the case, the two transcendental instructions cancel out and the scale becomes a linear function of the specular power.

The images below show the results using the warp filtering method (these were chosen to highlight the artifacts of the warp method). Hover the cursor over the images to visualize the results of the new approach:

I’d like to thank Sebastien Lagarde for his valuable feedback while testing these ideas and for providing the nice images accompanying this article.

Note: This article is also published at AltDevBlogADay.

The current puzzle count is 440.

As expected in the previous update, over the past week or so I went back and worked on the endgame again. I built an original version of the endgame last summer; I had one friend playtest it, and it worked out pretty well, though I felt it wasn’t quite epic enough. I didn’t know what more to do with it at the time, so I just let it sit, and went to thinking about other parts of the game.

Lately I had more ideas for the endgame, so I went into personal crunch mode and rebuilt into something more epic. This was one of the most productive periods of my working life (maybe the most productive) and I think this area, all taken together, may be the most interesting thing I have ever designed. However, it may be a little too difficult for the player, at the moment! Today I playtested this endgame and one other area of the game, with a game designer friend of mine, and it took nine and a half hours to play through these two areas (6 hours for the endgame, 3.5 hours for the other area). I think this is a little long, so I am going to be tuning these areas and cleaning them up a little bit. This weekend hopefully I can get in some playtest time with Jeff and Casey of Jeff and Casey Time, so if that happens we will see how it goes.

(It’s interesting; I haven’t done a playtest of The Witness for a while, but I was still thinking of it as a 10-15 hour game, and that’s what I tell people when they ask. But if these two areas by themselves, even after being cleaned up, are 6-7 hours of playtime, that means the whole game is … … much longer. Sometime in a month or two, I’ll do some playtests with people who have never seen the game before, and we’ll see how long it really is.)

Meanwhile, on the art side, we’ve been working on the style of the game; expect a future blog post as we come to conclusions on that.

Tech-wise, Andy has been working on performance stuff. First he made raycasting much faster; til now the game had been relatively lazy about spatial organization. Andy implemented a quadtree for finding entities in the world and enabled k-d tree generation and serialization for casting rays against individual meshes. (We had k-d tree code already as part of a library but weren’t using it.) Salvador is working on in-game animation playback, as well as exporting; until now we have been making do without animations, but once this is in the game it will help us with a number of issues.

Our gift to you, on this very special Valentine’s Day, is the worldwide release of the long-awaited game Dear Esther.

If you haven’t heard of Dear Esther, watch this:

Click here to view the video on YouTube.

(Or hey, watch the trailer even if you are quite familiar with the game; the trailer is beautiful and worthy of multiple viewings.)

We expect public reception of this game to run wide: some will love it, and others will be very concerned about whether this thing can be called a game and what that means. So far, this has certainly been the case in pre-release reviews.

Game Informer scored the game an 8/10, saying: “You should consider checking out Dear Esther the same way you’d appraise a film. If you’re interested in absorbing an intellectual story and gorgeous visuals without having to exert a drop of effort, take a chance on this curious experiment.”

VideoGamer.com scored the game a 9: “Discovery is such an important part of Dear Esther, especially when everything is so phenomenally pretty.”

Meanwhile, Destructoid gave the game a lowly 4.5/10: “It’s as if it wants to be a part of this wonderful medium of ours without asking itself why, which is exactly why you should seek it out and learn from its failures as a game enthusiast, critic, or developer.”

We like that there’s such a big difference of opinion because it means the game is breaking new ground. It’s playing in territory that is not safe; there is no established understanding there.

Dear Esther is a game that no publisher would have funded. Dan, Rob, Jessica and the other associates of thechineseroom have done an excellent job putting together a beautiful game. We are happy to be backing it; we hope you enjoy playing it.

If you’d like more information about Dear Esther, here’s an interview with Dan, and here’s a link the game’s page on Steam.

Update: Dear Esther has been wildly successful, selling 16,000 copies in under 24 hours. As an investment, it reached profitability in 5.5 hours. More details are available at a new post on the Indie Fund blog.

This is because this corner of the world is one of the last areas that we haven’t really figured out, in terms of what goes where. There is sort of just some random stuff dumped there right now. That will change pretty soon.

Expect the next few island snapshots to show a lot of big changes. We’ve reached the point where the basic gameplay is settled, I’m refining many of the specific puzzles, and we’ve got at least rough drafts of many of the architectural layouts. That means we can start planning the island a bit better. Over the next month or so, we’re going to be changing the shape of the island to something a little more geologically plausible, something that carries more information. It’s going to be interesting to do this without disrupting the gameplay layout of the island too much.

The current puzzle count is 427. In January I scrapped and re-designed an area and its 60-or-so puzzles. The previous area was inside dark caves, which is a bit cliche, and the gameplay hook that motivated my setting that area in caves, which I had hoped would be cool, didn’t work very well. In addition, I felt that the puzzles themselves, while fitting into the rest of the game, were the weakest part of the game. I’d been feeling this way for a long time, but I didn’t have a good solution for it til late December. I moved the location and redesigned the puzzles. Now the theme of the location is totally different, has a gameplay hook that works way better than the cave thing, and is very cool. The puzzles situated in this area are also much stronger than before, because I found a way to change the rules of those puzzles to make them much tighter. They’d felt wishy-washy before (people would play through that area and, after finishing, lack clarity on what the puzzles were saying. That’s no good.)

We certainly could have shipped the game with the old area in the caves with the old puzzles, and probably shipped the game at least a month sooner, but an important skill in building a beautiful game is knowing when something just isn’t good enough.

Next I am going back to the endgame, which I did my first pass at last summer sometime; I’ve got a gameplay hook to add which I think will bring it over the top and make it extra-good. I’ll be working on that over the next few weeks while also working with the landscape architects on the island revision.

Also, we’ve got a new programmer starting soon (this week or next!) Combined with our other new addition, who’s been here a few weeks (Andrew Smith, ex-Zipper and -Oddworld; hi Andy!) we will have quite a bit of tech manpower to bring to bear on finishing this game.

The trailer has just been released:

Click here to view the video on YouTube.

See more about Dear Esther at the official site.

This will be the second Indie Fund game to release. Just a few days ago we announced that Q.U.B.E., the first funded game, is already a financial success. It appears obvious that Dear Esther will be as well.

A small team with limited manpower, we wanted a low-effort way to use our existing version control system (svn) to facilitate concurrent world editing for our open-world 3D game (i.e. we desire that multiple users can edit the world at the same time and merge their changes).  We developed a system wherein the data for each entity is stored as an individual text file.  This sounds crazy, but it works well: the use of individual files gives us robustness in the face of merge conflicts, and despite the large number of files, the game starts up quickly.  Entities are identified using integer serial numbers.  We have a scheme for allocating ranges of these numbers to individual users so there is no overlap (though this scheme could certainly be better).  The system seems now to be robust enough to last us through shipping the game, with seven people and one automated build process concurrently editing the world across 18 different computers.

We don’t know whether this method would work for AAA-sized teams, though it seems that it might with some straightforward extensions.  But for us the method works surprisingly well, given how low-tech it is.

About the Game

We have been developing a 3D open-world game called The Witness since late 2008.  For much of that time, development happened in a preproduction kind of way, where not many people were working on the game.  Recently, though, we went into more of a shipping mode, so the number of developers has grown: we’re up to about 10 or 11 people now (pretty big for a non-high-budget, independent game).  Not all of these people directly touch the game data, but about 7 do now, and this number will rise to 9 in the near future.

The design of the game requires the world to be spatially compact, relative to other open-world games, so it is far from Skyrim-esque in terms of area or number of entities.  At the time of this writing, there are 10,422 entities in the game world.  I expect this number to grow before the game is done, but probably not by more than 2x or 3x the current amount.

Because the world is not too big, and some game-level events require entities to coordinate across the length of the world, it makes sense to keep all the entities in one space, that is to say, there is no zoning of the world or partitioning of it into separate sets of data.  This makes the engine simpler, and it reduces the number of implementation details that we need to think about when editing the world.  It also introduces a bit of a challenge, because it requires us to support concurrent editing of the world without an explicit control structure governing who can edit what.  One such control structure might be: if you imagine the world were divided into a 10×10 grid; so there are 100 different zones; and where anyone editing the world must lock whatever zones they want to edit, make their changes, check them in, then unlock those zones; this would provide a very clear method of ordering world edits.  (Because of the locking mechanism, two people would not be able to change the same world entity at the same time, so there doesn’t have to be any resolution process to decide whose changes are authoritative). However, the straightforward implementation of such a control scheme would make for inconvenient workflow, as it is tedious to lock and unlock things.  Even with further implementation work and a well-streamlined UI, there would still be big inconveniences (I really want to finish building a particular puzzle right now, but a corner of that puzzle just borders on some region that a different user has locked, and he is sick this week so he probably won’t finish his work and unlock it until next week.  Or, if my network connection is flaky then I may not be able to edit; if I am off the net then I definitely could not edit.)

Thus, rather than implementing some kind of manual locking, I found it preferable to envision a system that Just Magically Works.  Since we already used the Subversion source-control system to manage source code and binary source assets such as texture maps and meshes, it didn’t seem like too big of a stretch for Subversion to handle entity data as well.

The World Editor and Entity Data

Our world editor is built into the game; you press a key to open it at any time.  Here’s what it looks like:

The editor view is rendered in exactly the same way as the gameplay view, but you fly the camera around and select entities (I have selected the red metal door at the center of the scene, so it is surrounded by a violet wireframe).  In the upper right, I have a panel where I can edit all the relevant properties of this Door; the properties are organized into four tabs and here you can see only one, the “Common” properties.

These properties, which you can edit by typing into the fields of this Entity panel, are all that the game needs to instantiate and render this particular entity (apart from heavier-weight shared assets like meshes and texture maps).  If you serialize all these fields, then you can unserialize them and generate a duplicate of this Door.  That’s how the editor saves the world state: serialize every entity, and for each entity, serialize all fields.

Originally, the game saved all entities into one file in a binary format.  This is most efficient, CPU-performance-wise, and the natural thing that most performance-minded programmers would do first.  Unfortunately, though, svn can’t make sense of such a file, so if two people make edits, even to disjoint parts of the world, svn is likely to produce a conflict; once that happens, it’s basically impossible to resolve the situation without throwing away someone’s changes or taking drastic data recovery steps.

Our natural first step was to change this file from binary into a textual format.  Originally I had worries about the load-time performance of parsing a textual file, but this turned out not to be an issue.  I decided to use a custom file format, because the demands in this case are very simple, and we have maximum control over the format and can modify it to suit our tastes.

Many modern programmers seem to have some kind of knee-jerk inclination to use XML whenever a textual format is desired, but I think XML is one of the worst file formats ever created, and I have no idea why anyone uses it for anything at all, except that perhaps they drank the XML Kool-Aid or have XML Stockholm Syndrome.  Yes, there are XML-specific tools out there in the world, and whatever, but I didn’t see how any such tool would be of practical use to us.  One of our primary concerns is human-readability, so that people can make intelligent decisions when attempting to resolve revision control conflicts.  XML is not particularly readable, but we can modify our own format to be as readable as it needs to be in response to whatever real-world problems arise.  You’ll see a snippet below of the format we use.

Evolution of the Textual Format

The first version of our textual format was a straightforward adaptation of our binary format.  The gameplay-level code that saves and loads entities is the same for either format; the only difference is just a boolean parameter, choosing textual or binary, which is passed to the lower-level buffering code.  After the buffer is done being packed, it’s written to a file.

In our engine, entities are generally referenced using integer serial numbers (that act as handles).  These numbers are the same when serialized as they are at runtime.  Originally, in the binary file, the order in which the entities were written didn’t matter.  But if we want to handle concurrent edits, we want to make merging as easy as possible for the source control system, so we sort entities by serial number, writing them into the file in increasing order.  Thus we didn’t generate any spurious file changes due to haphazardly-changing serialization order.

In theory, Subversion could now handle simultaneous world edits and just merge them, the same way it does with our source code.  Unsurprisingly, though, this initial attempt was not sufficient.  Though the file was now textual, it contained a bare minimum of annotations and concessions to readability.  Though I knew I would eventually want to add more annotations, I didn’t want to go overboard if it wasn’t necessary, because these inflate the file size and the time required to parse the files (though I never measured by how much, so it’s likely the increases are negligible performance-wise and I was engaging in classic premature optimization).  As it happens, the kind of redundant information that increases human-readability also helps the source control system understand file changes unambiguously, so as we made the format more-readable, the number of conflicts fell.

Here’s what our file format currently looks like, for entity #6555 (the Door we had selected in the editor screenshot).

Door
72
6555
; position
    46.103531 : 42386a04
    108.079681 : 42d828cc
    8.694907 : 410b1e57
; orientation
    0.000000 : 0
    0.000000 : 0
    -0.069125 : bd8d9184
    0.997608 : 3f7f633d
; entity_flags
    160
; entity_name
    !
; group_id
    4059
; mount_parent_id
    6554
; mount_position
    0.035274 : 3d107ba5
    0.006069 : 3bc6e19c
    0.000038 : 381ffffc
; mount_orientation
    0.000000 : 0
    0.000000 : 0
    0.000027 : 37e5f004
    1.000000 : 3f800000

(The first line, “Door”, is the type name; 72 is the entity data version, which is used to load older versions of entities as their properties change over time; 6555 is the entity’s ID.  root_z and cluster_id (seen in the screenshot) are not saved into the file, because these properties have metadata that say they are only saved or loaded under certain conditions which are not true here.  For the sake of making the listing short, I’ve shown only the properties listed on that first tab in the screenshot.)
You’ll note that the floating-point numbers are written out a bit oddly, as in this first line under ‘position’:

    46.103531 : 42386a04

This is how we preserve the exact values of floating-point numbers despite saving and loading them in a text file.  If you do not take some kind of precaution, and use naive string formatting and parsing, your floating-point values are likely to change by an LSB or two, which can be troublesome.

The number on the left is a straightforward decimal representation of the value, written with a reasonable number of digits.  The : tells us that we have another representation to look at; the number on the right, which the : introduces, is the hexadecimal representation of the same number.  The IEEE 754 floating-point standard defines this representation very specifically, so we can use this hexadecimal representation to reproduce the number exactly, even across different hardware platforms.  The decimal representation is mostly just here to aid readability.  However, if we decide we want to hand-edit the file, we can just delete the : and everything after it, changing that first number to a 47.5 or something; seeing no :, the game will parse the decimal representation and use that instead.

There are other ways to solve this floating-point exactness problem; for example, you can implement an algorithm that reads and writes the decimal representation in a manner that carefully preserves the values.  This is complicated, but if you’re interested in that kind of thing, here’s what Chris Hecker says:

Everybody just uses David Gay’s code:
http://www.netlib.org/fp/dtoa.c
http://www.netlib.org/fp/g_fmt.c

You need to get the flags set right on the compiler for it to work. /fp:precise at least, I also do #pragma optimize (“p”,on), and make sure you write a test program for it with your compiler settings.

(Twitchy, and look at how much code that is!  What we did is a couple of lines.) In a document on his web site, Charles Bloom discusses a scheme that was in use at Oddworld Inhabitants, similar to what we’re doing in The Witness but involving a comparison between the two representations:

… Plain text extraction of floats drifts because of the ASCII conversions; to prevent this, we simply store the binary version in the text file in addition to the human readable version; if they are reasonably close to each other, we use the binary version instead so that floats can be persisted exactly.

Subversion Confusion

After switching to this format, conflicts still happened.  I categorize conflicts into two types: “legitimate”, where two people edit the same entity in contradictory ways and the source control system doesn’t know who to believe; and “illegitimate”, where Subversion was just getting confused.  We did have a number of legitimate conflicts for a while, yet which were not the users’ fault, because our world editor tended to do things that are bad in a concurrent-editing environment.  (See the Logistics section at the end of this article for some examples).  But even after we ironed these out, we still found that Subversion was still getting confused.

Here’s an example of a common problem: suppose we start with a world containing data on entities A, B, and D, so our file contains: ABD.  You edit the file and delete entity B.  I edit the file and also delete entity B, but add another entity C.  (That we both deleted B may be a very common use-case, especially if this is due to some automatic editor function).  So, you want to check in a file that just says: ABD – B = AD, so the new result is AD.  I want to check in a file that says: ABD – B + C = ACD (the C goes in the middle, because remember, the entities are sorted!)  In theory this is an easy merge: the correct result is just ACD.  But Subversion would get confused in cases like this, requiring the user to hand-merge the file.

This is a disaster, because even with a visual merge tool such as the one that comes with TortoiseSVN, you don’t want non-engine-programmers trying to merge your entity data file.  Even if that file is extremely readable, at some point someone will make a mistake; the file will fail to parse, and then you have a data recovery problem that requires the attention of an engine programmer.  When this happens it kills the productivity of two people (the engine guy, because he has to merge the data, and also the guy who generated the conflict, because until it gets resolved his game is in an un-runnable state, and anyway, the engine guy has probably commandeered the other guy’s computer to fix this).

After we had this problem a few times too many, Ignacio suggested moving to a more-structured file format like JSON, which, in theory, source-control systems would handle more solidly.  But I thought that this would only reduce the frequency of the problem, but not solve anything fundamental; even if we achieved a situation of only generating legitimate conflicts, each conflict would still be a potential game-crippling disaster.

Splitting the File

It occurred to me, finally, that it could hardly get cleaner than to store each entity in its own file.  This initial thought might cause a performance-oriented programmer to recoil in horror; we have 10,422 entities right now, and it must be very slow to parse and load 10,000 text files every time the game starts up!

But I knew that we already had over 4,000 individual files laying around for assets like meshes and texture maps, currently loaded individually, the idea being that at ship time we would combine these into one package so as to avoid extra disk seeks and the like.  But on our development machines, these files still loaded quickly.  I figured that if we can already load 4,000 big files, why not try loading an extra 10,000 small files?  If loading all these files turned out to be slow, we could resort to this only when entity data has changed via a source control update; after loading all the text files, we can save out a single binary file containing all the entity data, which would be much faster to load, just like we used to do before we supported concurrent editing.

I tried this and it worked great.  So, now we have a data folder called ‘entities’, which holds one file per serialized entity.  The Door selected in the editor screenshot is stored in a file named after its entity ID, “6555.entity_text”, which contains the text you see in the listing above.

As mentioned, there are currently 10422 entity files; the text files take up about 8MB of space, whereas the binary file is 1.4MB.  We’re not making any effort to compress either of these formats; they are very small compared to the rest of our data.

Benefits of Using Individual Files

Storing each entity in a separate file solved our problems with merging: both the illegitimate source control confusions as well as the problems due to hand-guided file merging.

The illegitimate conflicts go away because transactions become very clear.  If I delete an entity, and you delete an entity, then we both just issue a file delete command, and it’s very easy for the source control system to see that we agree on the action.  Furthermore, if we both delete B but I add C, because the change to C is happening in a completely separate file, there is no way for the system to get confused about what happened.

Legitimate conflicts do still happen, as multiple people edit the same entities, but these cases are much easier to handle than they were before.  The usual response is to either revert one whole entity or stomp it with your own.  Because we usually want to treat individual entities atomically — using either one person’s version of that entity or the other’s — individual files are a great match, because now the usual actions we want to perform are top-level source control commands (usually from the TortoiseSVN pulldown menu); before, this wasn’t true, and merging a single entity in an atomic way required careful use of the merge tool.  If an entity ever gets corrupted somehow, for whatever reason, we can just revert that one file, knowing the damage is very limited in scope; before we’ve reverted it, the worst that can happen is that that single entity will not load; it won’t prevent the rest of the entities in the world from loading.

It would be difficult to overemphasize the robustness gained.  I feel that these sentences do not quite convey the subtle magic; it feels a little like the state change when a material transitions from a liquid to a solid.

Sometimes, someone will update their game data in a non-vigilant way; not noticing that a conflict happened, so they won’t resolve it; and then they will run the game.  Before, this would have been a disaster; Subversion inserts annotations into text files to try to help humans manually resolve conflicts (this is an unfortunate throwback to programs like SCCS), which would have caused much of the world to fail to parse.  Now, though, the damage from these annotations is constrained to individual entities. It’s also easier to detect: if there is a conflict in the file for our Door #6555, then in addition to writing these annotations into 6555.entity_text (making it unparseable by the game), Subversion will also write out files 6555.entity_text.mine and 6555.entity_text.theirs, which contain the two intact but conflicting versions of the file.  The game can detect these very straightforwardly, put an error message on the HUD, and even choose one to load by default.  It is all very clean.

This is one of those interesting cases where reality-in-practice is starkly different from computer-science-in-theory.  In theory, a directory is just a data structure organizing a bunch of black-box data, and we happen to view this organization in a way that looks like a list of files; we could just do the same thing interior to our own file format, and it would be no big deal.  In practice, due to the way our tools work, these two arrangements are tremendously different.  The individual files just feel solid and reliable to work with, whereas one big structured file is always going to be precarious.

Startup Procedure

When the game starts up, we need to load all the entities from disk.  In the simplest case, this is straightforward: we just iterate through every file in the entities directory and load it.  However, as I mentioned, we wanted to keep a binary version of the entities that is preferred by the game if it exists and is up to date; partly, this might help startup speed a little (we don’t have to parse all these strings), but mainly, that is the file format and code path we are going to use when the game ships, and we don’t want that code to rot in the meantime.  Using it every day is the best way to prevent it from rotting.

So: when the game first starts up, we check for the binary entity package.  If it’s not there, we load all the individual entities files, then immediately save out the binary package.  (This binary package is a private, local file; if we were to add it to source control, everyone would always conflict with everyone else.)  But suppose we start up and the binary package is already there (which it usually will be).  We need then to determine if any of the textual entities have changed since the binary package was built.  The most-general way to do this is just by looking at the file timestamps; if we were to rely on some other signal being written to the disk that says “hey, entity data has changed,” then we would only be allowed to modify the entity files using sanctioned and customized tools; that would be annoying and would cause all kinds of robustness issues.

In concrete terms: we want to know the timestamp of the newest .entity_text file, inclusive of the directory containing these files (if we just did a Subversion update that only deleted entities, then all the .entity_text timestamps will be old, but the directory timestamp will be new; we want to ensure we properly rebuild the binary file in this case!)  To find the newest timestamp, we just iterate through all the timestamps in a brute-force way:

bool os_get_newest_file_modification_time(char *input_dir_name, Gamelib_Int64 *file_time_return) {
    //
    // We include the modification time of the directory as well.
    //

    char *dir_name = mprintf("%s/*", input_dir_name);
    Free_On_Return(dir_name);
    if (strlen(dir_name) >= MY_MAX_PATH - 5) return false;

    //
    // best_time starts as the time for the directory (if it exists!)
    //
    Gamelib_Int64 best_time = 0;
    bool success = get_file_modification_time(input_dir_name, &best_time);
    if (!success) {
        Log::print("Unable to get modification time for directory '%s'!\n", input_dir_name);
    }

    //
    // Now consider the file timestamps.
    //
    WIN32_FIND_DATA find_data;
    HANDLE handle = FindFirstFile(dir_name, &find_data);
    if (handle == INVALID_HANDLE_VALUE) return false;

    while (1) {
        if (find_data.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
        } else {
            Gamelib_Int64 file_time = to64(find_data.ftLastWriteTime);
            if (file_time > best_time) best_time = file_time;
        }

        BOOL success = FindNextFile(handle, &find_data);
        if (!success) break;
    }

    FindClose(handle);

    if (file_time_return) *file_time_return = best_time;

    return true;
}

bool get_file_modification_time(const char *name, Gamelib_Int64 *result_return) {
    HANDLE h = CreateFile(name, 0, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
                          NULL, OPEN_EXISTING, FILE_ATTRIBUTE_READONLY | FILE_FLAG_BACKUP_SEMANTICS,
                          NULL);
    if (h == INVALID_HANDLE_VALUE) return false;

    FILETIME write_time;
    GetFileTime(h, NULL, NULL, &write_time);
    CloseHandle(h);

    if (result_return) *result_return = to64(write_time);
    return true;
}

You’ll notice that this is Windows-specific code, but we don’t have to worry about porting it, because we only run the world editor on Windows.  Fortunately, in Windows, the file timestamp is right there in the directory data, so we don’t have to open each file.  (At least in NTFS this is true, rather than being an API illusion; the timestamps are stored in the Master File Table [MFT].)

What’s the performance like?  I typically run the game on two different machines: a high-performance Tower PC (CPU: Intel Core i7-970, hard drive: OCZ Vertex 3 SSD, RAM: 12GB) and a gaming Laptop (CPU: Intel Core i7-2820QM, hard drive: Intel SSD 500 [SSDSC2MH250A2], RAM: 12GB).

The time spent in os_get_newest_file_modification_time is:  Tower: .0095 sec, +/- .0015 sec; Laptop: .039 sec, +/- .005 sec.  (Timings were computed by averaging six trials, after a discarded warm-up run.)  This is definitely one of those cases where I’m glad I didn’t prematurely optimize, as I would have expected this to be much slower!

We typically install SSDs in all our work machines, but even when people have home machines that use spinning-platter hard drives, startup times are not much slower (likely due to the MFT data staying in cache).

If for a large enough number of entities this timestamp-gathering process becomes too slow, we can always write some code to read the timestamps out of the MFT directly, avoiding all the API call overhead, as in this forum posting.  To date we’ve had no need to do this.

Some people have reported performance problems when Windows has more than 100,000 files in one directory, but this can be worked around: just make 10 directories, with IDs ending with each digit from 0-9 grouped in common directories.  (Why use the least-significant “ending” digit, rather than the most-significant “starting” digit, which might be the impulsive choice?  Because if you do it that way, Benford’s Law says your entities are likely to be unevenly distributed across the directories.  Why not just hash them into a folder?  Because it’s useful to be able to manually navigate to the entity you want, without searching.  (However, partitioning by least-significant digit may be inconvenient if you want to manually edit a sequential range of entities for some reason, because that range will span many directories.  If this is an important use case, perhaps there’s a good compromise like using the tens digit or hundreds digit — though I have the itching feeling that a clearly superior solution would exist.))

As things stand in terms of editing the game world, we are probably not yet at the peak entity count, but there’s unlikely to be more than 3x the number of entities we have now.  Given how low the timing measurements are now, it’s clear that performance will not be a factor in the ability to use this system up until shipping the game.

Partitioning the ID Space

As I mentioned, in our engine, entity IDs are integer serial numbers.  We like this because they are easy to type (we have to type them in the editor sometimes) and at runtime, looking up an entity compiles to a fast indexing of a flat array, with a bounds check.

This introduces another problem for us to deal with in world editing, though: we need to ensure that two users, who are both creating new entities, do not both try to use the same number.  This would result in both users attempting to add completely incompatible .entity_text files with the same name.

One obvious and bulletproof solution is to have the editor connect to the network and atomically grab an ID from a central server each time one is needed.  But again, we don’t want to require users to be on the network in order to edit.

My solution was a somewhat lazy one: we allocate a range of IDs to each computer that edits the world.  These are kept in a file called “id_ranges”; at startup time, the game queries its machine’s MAC address, then looks in the file for the corresponding ID range; if not found, it claims a new range and adds a new line to the file:

 6000 10000 BC-AE-C5-66-28-E9 #Jonathan_office
10000 13000 BC-AE-C5-66-28-48 #Ignacio_office
13000 16000 00-1C-7B-75-00-D8 #Jonathan_laptop
16000 19000 BC-AE-C5-66-28-93 #Shannon_office
19000 22000 BC-AE-C5-56-42-51 #Jonathan_home
22000 25000 C0-3F-0E-AA-F3-F2 #icastano_home
25000 28000 F4-6D-04-1F-D4-36 #Shannon_home
28000 31000 Baker #Baker
31000 34000 BC-AE-C5-56-43-FE #Misc_PC_office
34000 37000 00-E0-81-B0-7A-A9 #eaa
37000 40000 00-21-00-A0-8F-E3 #Orsi
40000 43000 64-99-5D-FD-97-64 #Jonathan
43000 46000 08-86-3B-42-0B-8D #Misc
46000 49000 90-E6-BA-88-9B-B7 #andrewlackey
49000 52000 48-5B-39-8C-80-5E #eaa
52000 55000 50-E5-49-55-64-BF #facaelectrica
55000 58000 00-FF-D4-C7-15-A3 #facaelectrica
58000 70000 Clusters #Clusters
71000 75000 Overflow #Overflow
75000 78000 F4-6D-04-1F-CC-77 #Nacho
78000 81000 BC-AE-C5-74-7B-A3 #Thekla

On each line, everything after the # is just a comment telling us in a human-readable way whose machine is responsible for that range (it defaults to the Windows username, but we can hand-edit that if we want).

(“Baker” is for an automated process, so I made a special case that uses the string “Baker” instead of the MAC address, just to minimize the potential for the build machine to claim a new range without us noticing and failing to check that in; see below.  ”Clusters” is for an editor operation that is also run in an automated process, but which can also be done manually by anyone; cluster generation removes all previous clusters from the world, so if clusters didn’t have their own space, you would end up frequently deleting entities in other peoples’ ranges; this seems like it might be a bad idea, though maybe, in reality, it’s fine and there would be no problem.  But as it stands we know that anything in the Clusters range is completely disposable, so if a conflict ever arises there we can just revert or delete those entities without a second thought.)

This ID space is naturally going to have a lot of big holes in it, but before we ship we will renumber the entities so that we can pack them tightly into an array.  In the meantime, it is sparse but not pathologically so; we can still get away with using a linear array to index the entities, though it will use more memory now than at ship.

This id_ranges file is a bit of a lazy solution, and it causes a few annoyances.  The most annoying thing is that, any time someone runs the editor for the first time on a new machine, they need to immediately check in the id_ranges file to publish their allocation to the rest of the team; and they need to coordinate so that nobody else is trying to do the same thing.  Because the editor just allocates the next 3000 IDs starting with the highest ID in the file, if two people try to allocate IDs simultaneously, they will both claim the same range.  We could solve this by requiring the editor to hold a source control lock to modify the id_ranges file, but I just haven’t gotten around to it, because this hasn’t been a big-enough problem.  Furthermore, we could have the editor automatically commit the changes to the file; but currently, nothing is ever published to the rest of the team automatically — publishing always requires manual action, and this seems like it may be a good thing to preserve.

But, not having implemented these solutions, we have been taken by surprise a couple of times by interactions with a second lazy choice of this system: the fact that we file only records one MAC address per computer.

Most modern computers have multiple MAC addresses (for example: if your computer has a Wifi adapter and two Ethernet ports, it’s probably got at least 3 MAC addresses).  My lazy code just asks Windows to iterate the MAC addresses and reports the first one for use in the file.  Sometimes, though, the reported address will change, which causes someone’s machine to modify the id_ranges file when they didn’t expect it; we had this problem once when an audio contractor, who works offsite, installed some VPN software.  If that user is not vigilant about source control (which non-technical people often aren’t), it can be a while before this problem is discovered; meanwhile, that user has been creating entities that may conflict with someone else’s.

Most or all occurrences of this problem could be solved by recording all of the user’s MAC addresses in the file, instead of one; I just haven’t done so.

If we weren’t using integer serial numbers, we wouldn’t really need id_ranges at all.  For example, if we looked up entities at runtime by hashing unique string identifiers, then we would only need to ensure that each user has a unique string prefix.  We take a slight overkill approach, where each entity ID is a somewhat long string starting with the user’s MAC address; or we could even generate a Windows GUID for each entity.  MAC addresses are 48 bits wide.  If you concatenate the MAC address with a 4-byte integer, that’s just 10 bytes per entity ID, which isn’t much (we currently use 4, already!).  Of course encoding these as strings and storing them in a hash table is going to require more space, but it isn’t ridiculous.  Looking things up in a hash table all the time is slower than indexing a flat array, so we are sticking with the way we do things for now; but this would solve some problems and simplify the overall system, so it sounds like the kind of decision I would change my mind on later, as computers get even faster than they currently are.  So if you implement a scheme like this, you can ignore everything I said about id_ranges.  The biggest problem becomes making it easy to type longer entity IDs into the in-game editor.

Logistical Issues in Editor and File Format Design

In closing, I want to discuss changes we made to the editor and file format, in order to make them compatible with the paradigm of concurrent editing.

Arrays

Just about every game wants to serialize arrays into files.  When transitioning from a binary format, the most natural thing for a performance-minded programmer is to write the array length out first, then write each member of the array.  That way, when reading the file back, we know how much memory to allocate, and we avoid all the slow things one has to do when reading an array of unknown length.  However, encoding arrays in this way results in a textual format that always conflicts, because even when people are making compatible edits to an array, nobody is going to agree on what that array count should be; also, it becomes impossible to manually edit conflicts without typing the count in: if two people add elements to an array, the merged array will contain all the items, but neither version of the file will contain the correct count!

To solve this, of course, we go to the common text-file convention of using a syntactic designation that an array is now over, rather than pre-declaring how many elements there are.  This is obvious, but I mention it here because it is a relatively universal concern.

Automatic Changes

You will also want to minimize or work around cases where the editor automatically changes entity properties without user intervention, because often these will lead to conflicts.  A simple example from our game world: the sun is represented by a few entities that are instantiated in the world, a glowing disc and a couple of particle systems.  Every frame, the gameplay code positions these entities with respect to the current camera transform so that, when the player moves around the world, the sun will seem to be infinitely far away.  This happens in the editor too, because when editing we want to see the world exactly the way it will look in-game.  So, as soon as we enabled concurrent editing, everyone was checking in different coordinates for the sun (depending on where the camera happened to be when they saved!) and this caused a lot of conflicts.  My first solution was to add a hack that zeroes the position of sun entities when saving, but upon loading, this interacted badly with the way particle systems interpolate their positions.  Instead, I added an entity flag that signifies that an entity’s pose doesn’t matter, with respect to thinking about whether it has changed or not, so we will only modify the sun’s .entity_text files if some more-important properties are modified.  (This article is already long, but I’ll say that in general, we detect differences between an entity’s current state and its state when it was loaded, and we only write out .entity_text files when properties have changed.)

Relative Transforms

In our engine, all entities have ‘position’ and ‘orientation’ properties, which always represent the entity’s frame in worldspace, flat and non-hierarchically.  If we want an entity to be attached to another entity’s transform, we set the properties ‘mount_parent_id’, ‘mount_position’, and ‘mount_orientation’ (you can see all these properties in the listing for Door #6555 above); once per frame, the game engine computes the entity’s new worldspace transform based on its mount information, properly handling long dependency chains.  Once we started using textual entity files, we found that entities’ worldspace coordinates would often differ by an LSB or two, even when their parents had not moved (differences especially happening, for example, between Debug and Release builds of the game).  Even without concurrent editing, it’s good to clean this up as a matter of basic hygiene, because otherwise you’re checking in spurious changes all the time, increasing load on the source control system and making it harder for a human to see useful information in the change list.  I chose to solve this problem by making the mount-update code more complex: it detects whether an entity has moved of its own accord, and only updates child entities’ worldspace positions if the parent entity really moved (or if the child entity really moved!).

Terrain Editing

There is one piece of editing workflow that used to make sense before concurrent editing, but which now does not.  We have a terrain system wherein we can move control points around in worldspace, then select a terrain entity, which has a height-field mesh, and press a key to recompute that height field with respect to the new control points.  I originally made this recomputation happen only on manually-selected entities, rather than automatically on the entire world, for UI smoothness purposes: the recomputation takes some time.  But when you only recompute one block of terrain and not its neighbors, you get discontinuities in the terrain; the longer things go on, the more discontinuities you get.  Once in a while I would fix this by selecting all terrain entities in the world, recomputing them simultaneously.

A terrain discontinuity

The catch is that when terrain changes, it affects other entities.  We re-plant grass, so that the grass matches the new elevations; we change the positions of trees and rocks and other objects that are rooted to the terrain at a predetermined height.  Whenever you recompute some terrain, we collect all grass/trees/rocks within a conservative bounding radius, as these are all the things that might possibly change.  When dealing with small groups of terrain blocks, this already is a bit of a recipe for trouble (it is not too hard to overlap someone else’s edits this way); but it also means that if I recompute the entire terrain to eliminate those discontinuities, I am in turn editing all grass, trees, and rocks in the world, and will inevitably conflict with anyone else who has moved any of them. For now, we simply shy away from this kind of global operation; when you get a conflict in some entity that you probably didn’t care about, like grass, it is easy just to accept the remote version of the file.  Long-term, I expect some kind of heavier-weight solution would have to happen here, though I hope it does not involve much locking.

Issuing Source Control Commands

Our editor automatically issues source control commands when entities are created or destroyed, because it would be extremely tedious and error-prone to expect users to do this.  Initially we had some problems with source control commands unexpectedly failing, which quickly leads to chaos.  After some work, issuing source control commands seems to work with full consistency, but we are still a little wary of it; so we don’t, for example, want to issue the commands via a background process, adding more complexity to the system.  The problem with this is that svn commands are a bit slow.  Sometimes when I have been editing the world for a while, it can take 15 to 30 seconds to save my changes, which would have happened instantaneously without the svn commands.

Conclusion

We’ve put together a system that lets us concurrently edit an open world, using source control to manage our world edits.  It works well enough for us to ship the game, so I am happy with it.  If the scheme ever fails in some way, or requires heavy modification, I will post an update!

When we sat down to start working with the architects, one of the first things we did was put together a history of the island where The Witness takes place: how long has the island been there?  What groups of people lived there, and what kind of structures did they build?  How did later civilizations use the structures built by earlier groups?

Quickly we developed a running theme of sites that are old and have over time been used in two or three different ways.  When entering a new location, you probably first notice the most basic elements of its structure: where you can walk to easily, versus where you can’t; what puzzles or goals are calling for your attention, and what walls or doors might be preventing you from getting to them.  If this is all you care about, you can then play through the area in a very utilitarian fashion.  But if you have an eye for detail, you may notice the elements of modern construction shoring up the design of an older structure with a different purpose.  The design of the new structure tells you something about what the newer guys were doing there, and how it differs from what the older guys were doing there.  And if you look really closely, you can see traces of the original footprint of the site (perhaps these are very subtle, some bricks just rising above the dirt in a few places), and infer the purpose of this very old site too.

If you see the different civilizations that came to this island as embodying different philosophies; and you see the structures they built as representative of the way these philosophies led them to interact with the world; and you see further that when they replaced a site, it represents the rejection of some older worldview that they consider no longer useful, then perhaps you start to get some idea of the amount of backstory that can be encoded into the world, nonverbally.

All of this backstory is also directly relevant to the main story — who you are, why you are on this island, how you may get back home.  So, to put it another way, the game is constructed so that the more you pay attention to tiny details during your travels, the more insight you will have to the central story, even though it may not be obvious at any given time what a particular detail has to do with that story.

The most-recent issue of Game Informer printed an article that showed some of the first screenshots of the newly-architected areas appearing in the game engine.  It is an interesting article; pick up the issue if you haven’t!  They also put up a companion article online (but which not as extensive) here:

(Summary article at Game Informer)

The area featured in this article is the one that we have taken furthest so far in terms of modeling and texturing.  Here are some other shots of it:

The Keep, before and after

The Keep, close-up 1

The Keep, close-up 2

The Keep, close-up 3

We’ve done basic planning for some of the other areas, so I thought it would be nice to show some screenshots of work in progress.  For each of these, there’s a ‘before’ version (when we had a fully-playable game, but basic placeholder structures) and after (once we have incorporated the architects’ designs, revised the site plans and in some cases even the puzzles, etc):

 

The Compound, before and after

Glass Factory, before and after

Peninsula, before and after

Vault, before and after

Don’t regard these “after” images as representative of what the game will look like!  They are still works in progress, and the final game will look substantially better: scenes like these will feel richer due to the placement of detail objects, further care put into the texture mapping and lighting, refinement of the still-rough-drafty terrain features around these structures, and many other bits of subtle game development magic.  But I wanted to show these now since they provide a good taste of the game’s flavor, and they show how much structural intelligence is brought to the table by working with the architects.  Yes, we could have started with the placeholder structures and made them more elaborate and better-looking, in a general video-game-level-design way, but that’s different from having well-thought-out ideas subtly embodied in the structures of the areas, which is what we are going for.

Click here to view the video on YouTube.

Q.U.B.E. will be available on Steam.

(Web site for the game)

(Indie Fund announcement page)

Eventually, these things have to be figured out. Over the past month or so we’ve done some of that figuring. We nailed down the architecture for a couple of areas (including the all-important area where the player starts the game). We’ve done this in a way where the architecture and the puzzle ideas have synergy, which is always nice. In particular, there was one location that, from the beginning of development, was basically a big box with a window cut into it and some puzzles stuffed inside. For a long time (maybe 2 years!) I was never sure what to do with it, but eventually the architects suggested an idea for that location. I mocked it up and didn’t like it. They suggested another one. This one sparked an idea for how to make the structural properties resonate with the puzzle theme, and when I built even just a rough-draft of the building, I found the result to be strong. Lately we’ve been finalizing that location too.

So anyway, there are all these spots on the island that represented vague ideas, with puzzles there that were just in rough form, or that weren’t even implemented fully, but just served as reminders to do something there. I’ve been going through and filling in a bunch of those.

I have also been working with some other puzzles, which seemed “done enough to ship”, and thinking hard about how to make them better. Braid benefited from a lot of this kind of attention — a lot of games would have shipped with just early versions of the puzzles in that game, considering them to be just fine; but the more I worked with the puzzles in Braid, the more I found better, simpler, more-to-the-point versions of them. The same thing is happening here (though The Witness is a much bigger, more-complicated game, so sometimes it is challenging to get to that same kind of clarity.)

With regard to the visuals (modeling and texturing), things have been coming along nicely as well. Pretty soon a high-profile game magazine will publish some screenshots of one of the most-finished areas of the game, showing the look we are targeting and giving a strong hint of what the game will be like stylistically. When the magazine is out there we will link it here.

(This has not been a particularly coherent update; just wanted to let you all know what is up!)

Any system of interactivity can of course be explored: If X happens, what are the consequences? What are all the ways in which pattern Y expresses itself, and to what do those expressions lead? By inspecting the structure of a system in this way, we can find the core ideas of the system, and see how those ideas illustrate fundamental truths of our universe. We present a game design aesthetic that values looking for systems that express these truths in the cleanest possible way. We explain how this is different from more-traditional combinatoric design techniques; we show examples from our games and describe a method for applying the aesthetic in general.

Here’s a link to the Indiecade page, on which there’s also a presentation by Richard LeMarchand, and more to come.

Thanks very much to Ida C. Benedetto for recording the talk!

I’ll post something more concrete before too long, but I wanted to at least say something, since it’s been a while.

Team-wise, we have just hired two people (a 3D artist and a programmer), and we plan to hire another programmer as well, once his visa clears and he can get into the country. This is a pretty big staffing-up, percentage-wise, and will allow us to make things quite nice. The new folks will be starting over the next couple of months (they need to wind down the things they are working on currently!)

The current puzzle count is 350.

Some parts of the island are pretty messed up in this screenshot. For the press tour, we made sure the whole game was more-or-less solid and playable. Just after that, though, we were free to tear things apart, move things around, change the sizes of areas, add new areas, and all that. We’re still in the middle of that process and not everything has been made playable again.

Here’s the link.

The difference is not too pronounced in this scene, but with smoother-colored textures it becomes a lot more obvious (and I do want to use more smoother-colored textures, art-direction-wise!)

The situation is, the lightmaps are HDR (for some definition of “high”), so we needed a way to encode those values reasonably into a bitmap. We were using RGBM where M is just an overall brightness factor; you read RGB from the texture then multiply by M to get the output color. We used to just encode this naively, but the problem is, when M is less than 1 then it actually kills precision in your RGB, so the solution is, well, don’t ever let M go below 1; just clamp it. This is not rocket science and in fact is the kind of thing that is a little funny, as one starts wondering why we didn’t think of that a long time ago, but hey, that’s how it is sometimes. The results made me happy, though, so I am posting them.

The improved image quality may enable us to have higher dynamic range throughout the game (this is probably something we will evaluate later on), which would be very nice!

Previews of the game are now being published, and readers of this blog have been asking for gameplay details for a while. So it seems like a good idea to list the previews here. I’ll keep updating this post as I find more.

on Kotaku, by Stephen Totilo
on Joystiq, by Ben Gilbert
on Gamespot, by Sophia Tong
on IGN, by Anthony Gallegos
on Gamasutra, by Christian Nutt
on 1UP, by Matt Leone
on GamesRadar, by Charlie Barratt
on Giant Bomb, by Ryan Davis
on Giant Bomb, by Patrick Klepek (interview-ish)
on Time Techland, by Evan Narcisse
on IFC, by Evan Narcisse (interview)
by Dan Griliopoulos on Rock, Paper, Shotgun

interview with GamesRadar
interview with Cory Cohen of OXM
long interview with Dan Griliopoulos of Rock, Paper, Shotgun (contains a bunch of stuff not elsewhere)

interview with Christian Donlan of Eurogamer

(Sometimes these previews, from the way they are worded, make it seem like I made the entire game with my own bare hands. That is not the case; it’s just how this kind of article tends to sound. There are a number of people working on this game, who have creative input in various ways; see the About page for a partial list!)

• Lightmap Parameterization
• Hemicube Rendering and Integration

Unfortunately, by the time I was done with these, the work I had done on the gradients was not so fresh anymore, which is why it has taken me so long to complete this article.

In my previous post I mentioned the need to find an algorithm that would allow us to extrapolate the irradiance to fill lightmap texels with invalid samples, to smooth the resulting lightmaps to reduce noise, and to sample the irradiance at a lower frequency to improve performance. As some avid readers suggested Irradiance Caching solves all these problems. However, at the time I didn’t know that, and my initial attempts at a solution were misguided.

Texture Space Approaches

I had some experience writing baker tools to capture normal and displacement maps from high resolution meshes. In that setting it is common to use various image filters to fill holes caused by sampling artifacts and extend sampled attributes beyond the boundaries of the mesh parameterization. Hugo Elias’ popular article also proposes a texture space hierarchical sampling method, and that persuaded me that working in texture space would be a good idea.

However, in practice texture space methods had many problems. Extrapolation filters only provide accurate estimates close to chart boundaries, and even then, they only use information from one side of the boundary, which usually results in seams. Also, the effectiveness of irregular sampling and interpolation approaches in texture space is greatly reduced by texture discontinuities and did not reduce the number of samples as much as desired.

After trying various approaches, it became clear that the solution was to work in object space instead. I was about to implement my own solution, when I learned that this is what irradiance caching was about.

Irradiance Caching

This technique has such a terrible name that I would have never learned about it if it wasn’t due to a friend’s suggestion. I think that a much more appropriate name would be adaptive irradiance sampling or irradiance interpolation. In fact, in the paper that introduced the technique for the first time Greg Ward referred to it as lazy irradiance evaluation, which seems to me a much more appropriate name.

There’s plenty of literature on the subject and I don’t want to repeat what you can learn elsewhere. So, in this article I’m only going to provide a brief overview, focus on the unique aspects of our implementation, and point you to other sources for further reference.

The basic idea behind irradiance caching is to sample the irradiance adaptively based on a metric that predicts changes in the irradiance due to proximity to occluders and reflectors. This metric is based on the split sphere model, which basically models the worst possible illumination conditions in order to predict the necessary distance between irradiance samples based on their distance to the surrounding geometry.

Interpolation between these samples is then performed using radial basis functions, the radius associated to each sample is also based on this distance. Whenever the contribution of the nearby samples falls below a certain threshold a new irradiance record needs to be generated, in our case this is done by rendering an hemicube at that location. In order to perform interpolation efficiently the irradiance records are inserted in an octree, which allows us to efficiently query the nearby records at any given point.

For more details, the Siggraph ’08 course on the topic is probably the most comprehensive reference. There’s also a book based on the materials of the course; it’s slightly augmented, but in my opinion it does not add much value to the freely available material.

The Physically Based Rendering book by Matt Pharr & Greg Humphreys is an excellent resource and the accompanying source code contains a good, albeit basic implementation.

Finally, Rory Driscoll provides a good introduction from a programmer’s point of view: part 1, part 2.

Our implementation is fairly standard, there are however a two main differences: First, we are sampling the irradiance using hemicubes instead of a stratified Montecarlo distribution. As a result, the way the we compute the irradiance gradients for interpolation is somewhat different. Second, we use a record placement strategy more suited to our setting. In part 1 I’ll focus on the estimation of the irradiance gradients, while in part 2 I’ll write about our record placement strategies.

Irradiance Gradients

When interpolating our discrete irradiance samples using radial basis functions the resulting images usually have spotty or smudgy appearance. This can be mitigated by increasing the number of samples and increasing the interpolation threshold, which effectively makes them overlap and smoothes out the result. However, this also increases the number of samples significantly, which defeats the purpose of using irradiance caching.

One of the most effective approaches to improve the interpolation is estimating the irradiance gradients at the sampling points, which basically tell how the irradiance changes when the position and orientation of the surface changes. We cannot evaluate the irradiance gradients exactly, but we can find reasonable approximations with the information that we already have from rendering an hemicube.

In my previous article I explained that the irradiance integral was just a weighted sum over the radiance samples, where the contribution of each sample is the product of the radiance through each of the hemicube texels , the solid angle of the texel , and the cosine term . That is, the irradiance at a point is approximated as:

The irradiance gradients consider how each these terms change under infinitesimal rotations and translations and can be obtained by differentiating .

The radiance term is considered to be constant, so the equation reduces to:

In reality, the radiance is not really constant, but the goal is to estimate the gradients without using any information in addition to what we have already obtained by rendering the scene on a hemicube. So that we can improve the quality of the interpolation by only doing some additional computations while integrating the hemicube.

Rotation Gradient

The rotation gradient expresses how the irradiance changes as the hemicube is rotated in any direction:

The most important observation is that as the hemicube rotates, the solid angle of the texels with respect to the origin remains constant. So, the only factor that influences the gradient is the cosine term:

With this in mind, the rotation gradient is simply the rotation axis scaled by the rate of change, that is, the angle gradient:

And the rotation axis is given by the cross product of the z axis and the texel direction :

Therefore:

This can be simplified further by noting that is the length of before normalization and that results in a very simple expression:

Finally, the rotation gradient for the hemisphere is the sum of the gradients corresponding to each texel.

The resulting code is trivial:

foreach texel, color in hemicube {
    Vector2 v = texel.solid_angle * Vector2(texel.dir.y, -texel.dir.x);
    rotation_gradient[0] += v * color.x;
    rotation_gradient[1] += v * color.y;
    rotation_gradient[2] += v * color.z;
}

Translation Gradients

The derivation of the translation gradients is a bit more complicated than the rotation gradients because the solid angle term is not invariant under translations anymore.

My first approach was to simplify the expression of the solid angle term by using an approximation that is easier to differentiate. Instead of using the exact texel solid angle like I had been doing so far, I approximated it by the differential solid angle scaled by the constant area of the texel, which is a reasonable approximation if the texels are sufficiently small.

In general, the differential solid angle in a given direction is given by:

For the particular case of the texels of the top face of the hemicube, n is equal to the z axis. So, the differential solid angle is:

We also know that:

which simplifies the expression further:

Using this result and factoring the area of the texels out of the sum we obtain the following expression for the gradient:

The remaining faces have similar expressions, slightly more complex, but still easy to differentiate along the x and y directions. We could continue along this path and we would obtain closed formulas that can be used to estimate the translation gradients. However, this simple approach did not produce satisfactory results. The problem is that these gradients assume that the only thing that changes under translation is the projected solid angle of the texels, but that ignores the parallax effect and the occlusion between neighboring texels. That is, samples that correspond to objects that are close to the origin of the hemicube should move faster than those that that are farther, and as they move they may occlude nearby samples.

As you can see in the following lightmap, the use of these gradients resulted in smoother results in areas that are away from occluders, but produced incorrect results whenever the occluders are nearby.

Left: Reference. Middle: Irradiance caching without gradients. Right: Irradiance caching with flawed gradients

The hexagonal shape corresponds to the ceiling of a room that has walls along its sides and a window through which the light enters. The image on the left is a reference lightmap with one hemicube sample per pixel. The image on the middle is the same lightmap computed using irradiance caching taking only 2% of the samples. Note that the number of samples is artificially low to emphasize the artifacts. On the right side the same lightmap is computed using the proposed gradients. If you look closely you can notice that on the interior the lightmap looks smoother, but close to the walls the artifacts remain.

This is exactly the same flaw of the gradients that Krivanek et al proposed in Radiance Caching for Efficient Global Illumination Computation, but was later corrected in the followup paper Improved Radiance Gradient Computation by approaching the problem the same way Greg Ward did in the original original Irradiance Gradients paper.

I then tried follow the same approach. The basic idea is to consider the cosine term invariant under translation so that only the area term needs to be differentiated, and to express the gradient of the cell area in terms of the marginal area changes between adjacent cells. The most important observation is that the motion of the boundary between cells is always determined by the closest sample, so this approach takes occlusion into account.

Unfortunately we cannot directly use Greg Ward’s gradients, because they are closely tied to the stratified Montecarlo distribution and we are using a hemicube sampling distribution. However, we can still apply the same methodology to arrive to a formulation of the gradients that we can use in our setting.

In our case, the cells are the hemicube texels projected onto the hemisphere and the cell area is the texel solid angle. To determine the translation gradients of the texel’s solid angle is equivalent to computing the sum of the marginal gradients corresponding to each of the texel walls.

These marginal gradients are the wall normals projected onto the translation plane, scaled by the length of the wall, and multiplied by the rate of motion of the wall in that direction.

Since the hemicube uses a gnomonic projection, where lines in the plane map to great circles in the sphere, the texel boundaries of the hemicubes map to great arcs in the hemisphere. So, the length of these arcs is very easy to compute. Given the direction of the two texel corners , the length of their hemispherical projection is:

The remaining problem is to compute the rate of motion of the wall, which as Greg Ward noted has to be proportional to the minimum distance of the two samples.

The key observation is that we can classify all the hemicube edges in wo categories: Edges with constant longitude ( is invariant), and edges with constant latitude ( is invariant).

In each of these cases we can estimate the rate of change of edges with respect to the normal direction by considering the analogous problem in the canonical hemispherical parametrization:

For longitudinal edges we can consider how changes with respect to motion along the x axis:

and for latitudinal edges we can consider how changes with respect to motion along the y axis:

As we noted before, the rate of the motion is dominated by the nearest edge, so r takes the value of the minimum depth of the two samples adjacent to the edge.

A detailed derivation for these formulas can be found in Wojciech Jarosz’s thesis, I personally found his detailed explanation very didactic, but unfortunately his thesis is not freely available online.

Note that these are just the same formulas used by Greg Ward and others. In practice, the only thing that changes is the layout of the cells and the length of the walls between them.

As can be seen in the following picture, the new gradients produce much better results. Note that the appearance on the interior of the lightmap is practically the same, but closer to the walls the results are now much smoother. Keep in mind that artifacts are still present, because the number of samples is artificially low.

Left: No gradients. Middle: Flawed translation gradients. Right: Improved translation gradients.

In order to estimate the translation gradients efficiently we precompute the direction and magnitude of the marginal gradients corresponding to each texel edge without taking the distance to the edge into account:

// Compute the length of the edge walls multiplied by the rate of motion
// with respect to motion in the normal direction
foreach edge in hemicube {
    float wall_length = acos(dot(d0, d1));
    Vector2 projected_wall_normal = normalize(cross(d0, d1).xy);
    float motion_rate = (constant_altitude(d0,d1) ?
        -cos_theta(d0,d1) :      // d(theta)/dx = -cos(theta) / depth
        -1 / sin_theta(d0,d1);   // d(phi)/dy = -1 / (sin(theta) * depth)
 
    edge.translation_gradient = projected_wall_normal * wall_length * motion_rate;
}

Then, during integration we split the computations in two steps. First, we accumulate the marginal gradients of each of the edges scaled by the minimum edge distance:

foreach edge in hemicube {
    float depth0 = depth(edge.x0, edge.y0);
    float depth1 = depth(edge.x1, edge.y1);
    float min_depth = min(depth0, depth1);
 
    texel[edge.x0, edge.y0].translation_gradient -= edge.gradient / min_depth;
    texel[edge.x1, edge.y1].translation_gradient += edge.gradient / min_depth;
}

Once we have the per texel gradient, we compute the final irradiance gradient as the sum of the texel gradients weighted by the texel colors:

foreach texel, color in hemicube {
    Vector2 translation_gradient = texel.translation_gradient * texel.clamped_cosine;
    translation_gradient_sum[0] += translation_gradient * color.x;
    translation_gradient_sum[1] += translation_gradient * color.y;
    translation_gradient_sum[2] += translation_gradient * color.z;
}

Conclusions

Irradiance Caching provides a very significant speedup to our lightmap baker. On typical meshes we only need to render about 10 to 20% of the samples to approximate the irradiance at very high quality, and for lower quality results or fast previews we can get away with as few as 3% of the samples.

The use of irradiance gradients for interpolation allows reducing the number of samples significantly, or for the same number of samples produces much higher quality results. The following picture compares the results without irradiance gradients and with them, in both cases the number of samples and their location is the same, note how the picture at the bottom has much smoother lightmaps.

I think that without irradiance caching the hemicube rendering approach to global illumination would have not been practical. That said, implementing irradiance caching robustly is tricky and a lot of tuning and tweaking is required in order to obtain good results.

Note: This article is also published on #AltDevBlogADay.

Here’s some screenshots I just took. Here is a group of trees without a lightmap. As you can see, the underside looks pretty flat and boring, and it’s hard to see which limbs are in front of others.

Here’s the lightmap just by itself. While you can see a few lighting artifacts (like where the cross trees cross), overall the trees look pretty good even without diffuse textures.

And here’s the final. As you can see, the underside has a lot more definition to it.

Once I get some LOD’s I like, I’ll write up another update.

Sun beams. These are accomplished with a radial blur centered on the sun, during postprocessing. The nice thing about this effect is how dynamic it is — as you walk beneath the trees, the shape and intensity of the beams are changing all the time.

Due to the way the effect works, sometimes beams are drawn that are a little bit unphysical. You probably have to be looking closely to see that, but I am a little bit uncomfortable with it given that the game is all about the opposite of that. We’ll see how this evolves. We’ve been talking about augmenting this with other effects.

A revised water reflection effect.

Softening of the shoreline, so that it doesn’t look like one big alpha-blended water polygon intersecting the ground.

Here’s what the island looks like now:

It’s fun to look at this compared to another shot of the island, say, from about a year ago:

The current puzzle count is 328.

The talk’s about 38 minutes long, with about 40 minutes of Q&A afterward. Here’s a slideshow:

CSUA talk

If the above embedded player doesn’t work for you, or you prefer to download the files directly, here they are:

audio track
slides (ppt format)

I’ve worked on trees for games over the years, and I’ve come to a conclusion: There’s no good way to make them currently. Trees reject the idea of low poly versions of themselves by their very nature, and what you’re left with is many unsatisfactory ways of making them. It’s up to the artist to pick which one of these techniques to use and to do their best to make it work. That choice depends on a lot of things. Does it have to be real-time, or can it be pre-rendered? Where will the player be when he sees the tree? Up close, far away, underneath, from above, or (in the worst case) all of these? Will there be a few trees or a forest of them? Then there’s the engine constraints.  Overdraw, poly counts, texture space, LOD systems, lighting and light maps; all these affect how the trees need to be created. And none of that is even scratching the surface on the style you want the trees to have.

Still, I like a challenge as much as the next person.

Here’s an early visual mockup of the game that contains some trees. I didn’t actually make this, but it did give me some direction is where to go. As you can see, the trees are very colorful and full of volume. I wasn’t sure how close I could get to this, but I was going to give it a try. The problem with most low-poly game trees is that they are made with large sheets of polygons with branch and bough textures on them…and while this is certainly a way of making game trees, they usually can’t be considered volumetric.

The current trees I have right now started from a lucky accident. Early on the engine couldn’t handle alpha maps, so I want to make a bough structure that would look vaguely leafy without being too many polys and without using opacity maps. That’s how I came up with this.

It’s basically a ton of triangles sitting in a bowl. The bowl is there to simplify the bottom and hide the triangle nature of the triangles. It worked fine for the time I needed it. Recently I started adding a bunch of trees to the world and I wanted something fairly lightweight for even the large trees (the one I had done just before that was 40,000 polys, which was way too big). So I went to the old non-opacity tree, added some opacity maps and tried it out. And, surprisingly, it worked really well. It was giving me a volume I didn’t expect, and it looked really good at a distance.

Not only that, I took a look at them with only the lightmaps on and the structure looked really nice…especially the branches underneath.

They were still too heavy though, being 12 thousand polys. Worse, they looked really bad up close and you could see that, what read as a nice, thick foliage from a distance, was really a mess of triangles.

So the past few days I’ve been trying to make the trees look better from a close distance and be lower poly without losing what I liked about the previous ones. After a bit of struggle, I got something that seems to do the trick. Ignacio helped a lot, making a shader that made polys disappear when they became too edge on to the camera, which eliminated many ugly artifacts. The foliage is half the polys of the other trees, plus it looks a lot better close up. Here’s a close shot of it, plus a comparison shot with an older tree.

I still have a bunch of other ideas I want to add to these to make them look better, but I think we’re on the right track to getting trees that really do have a nice amount of volume and harken back to the early mockups.

But we also figured, hey, why not play around with the color of sunlight as well? The idea is that as you walk around in the game, the color of the sun and ambient light will interpolate as you move. We haven’t hooked it up that way yet; what you see below is just done by tweaking global variables in the game. Well be doing more experiments in this direction…

Much denser and more visually interesting; it’s starting to look like maybe a good game!

Here are some close-ups of different areas now.

Autumnal:

Desert:

Foresty (there is a lightmapping problem on some of these trees so they are brighter than they should be!):

But now we are thinking about how to create colorful scenes, like in the previous post, and this style of blend-mapping did not seem like a very good idea. In order to create gradients across the scene, we would need to have a lot of blend layers; this would be slow and use a lot of texture memory (since now we are storing a lot of individual blend maps). Even once we did that, it would be laborious to control, and it’s unclear that it would look good.

Ignacio mentioned that the Wolfire guys were just dropping one color map across their whole terrain. That seemed like a good idea to me — I have done that before, but in contexts where we were trying to really crank up the resolution to levels that were ridiculous at the time, like 8192×8192 texture maps back in 2001. Rather than doing that, the idea now is to just use what is a reasonable-sized texture on a modern machine, and see what it looks like. So Shannon painted a 2048×2048 texture of the island (of which we are only really using less than half right now) and here’s how it looks from up high:

Immediately this strikes me as much nicer, and much more in-tune with the art direction we want for the game. The buildings and other meshes now stick out like sore thumbs, but that is just what happens when you improve only a part of the scene. We’ll be working on this.

Here’s a scene from closer in:

As you can see, we have less detail close-up on the ground; things look a bit blurry. However, the colors are a lot better, and I think if we can adapt this scheme into something that permits sharper details up close, we will be able to make something that has a lot of character and looks special. In these shots, we are still using the old blend map system to tile high-detail normal maps across the terrain; that helps, but by itself it’s not enough. (In recent videos for Overgrowth, like this one, you can see that they are doing something that provides close-up details, maybe just intensity maps that are being modulated by the color map). So we’ll keep working on this and see how it goes.

Meanwhile, Ignacio has been revising the shadow map rendering. We used to have very speckly shadow maps, as I was using a 4-sample Poisson disc to sample the shadow map. Ignacio recently switched this to a Gaussian that has more taps, then applied some math tricks to reduce the number of taps. This results in softer shadows:

At some angles, shadow map jaggies still show up pretty clearly:

This is something we might work around in the final game by changing the scene when this happens! Or, it’s possible the shadows will be modified further so that they look smoother in cases like this. Who can say?

Shannon did a few color studies by taking screenshots of the game world as it stands today, then painting over them to build scenes with different palettes. Here’s what he came up with:

I think it looks quite nice. Don’t assume much based on the painterly style of these images; they were intentionally done at a low level of detail, so that we could quickly get a feel for the colors. We don’t know yet exactly what the rendering style of the game will be in the end (but that is something we will start working on soon!)

The puzzle count is now 285.

This means I am getting closer to my Gameplay Complete milestone. The bulk of the game design is now more-or-less there, in the sense that you can go and solve all the puzzles and they are basically the body of puzzles that will be there for the final game (though in the final game they will be refined, and more will be added — but if we had to ship with only the puzzles that are in there now, we could do that, and it would be a good game). However, there’s no endgame or winning condition, so I will be working on this next.

I think the endgame is going to be a bunch of work — the original idea was ambitious, and I need to do a bunch of redesign on it since the game as a whole has evolved since the time the endgame was originally conceived. We’ll see how it goes.

The visuals are still placeholder. In a couple of weeks we have some architects coming in to start designing buildings and terrain; at that time our production pipeline will change and we’ll be moving a little closer to “the real visuals”.

See the video here.

Thanks to blog commenter justin for pointing this out!

As a result of these sessions it immediately became clear how to redesign some of the areas to improve pacing. So I started radically restructuring three of the major areas, as well as rearranging the relative positions of areas on the map. That process isn’t done yet, but it is starting to converge (just one more area left to redesign). I am very happy with the two areas already redesigned; they are obviously far better than the original versions. It feels very rewarding when you feel good about a game already, and then do something to improve it so much.

Also, I’ve shrunk the island again. Here’s what it looks like now:

The current puzzle count is 275.

Also, we are hiring.

We at Thekla, Inc. are a small independent developer making games with tradition-breaking designs.  We are pushing the boundaries of what a video game can be. Our current project is The Witness, a puzzle/exploration game designed by Jonathan Blow, designer/programmer of the critically-acclaimed Braid.  Our office is in Berkeley, California, USA but we will consider remote work for the right candidates.

Senior Game Programmer (full-time)

Work with other excellent programmers to build technically-sophisticated games for a variety of platforms, including consoles, PCs, and tablets.  Work on systems-level infrastructure as well as gameplay code.  You get a lot of freedom to approach problems the way you want to, and to employ your full range of technical creativity.  Management overhead is minimal.  There are many perks you won’t find in other jobs.

Our code is mainly in C++.  When it comes to building software, we have a no-nonsense approach.

Job Requirements:

• Be good at making video games.

Senior Game Programmer (part-time, contract)

Like the job above, but part-time.

To inquire about any of these jobs, please send email to jon@number-none.com.

Here’s what it looks like now:

It does not look shipping-quality, but then, almost nothing in the game does! As the first results obtained by just slapping some pre-existing grass meshes down on the terrain without trying too hard, I find it highly encouraging.

Note that grass doesn’t cast shadows right now. There are two reasons for that: rendering speed and aliasing of the shadow map. I suspect we will try to make grass cast shadows before we ship, though, and if we make something good happen there, I think it will really boost the visual quality.

The one piece of code that pushed the grass over the threshold into general usability is that it now has the ability to sample terrain blend maps. (The terrain uses a blend shader, where a non-visible blend texture map controls the degree of mixing of different terrain materials. This is becoming a common thing in 3D games; Uncharted 2 and Crysis 2 do it, among others.) The grass can now peek at the blend map and reduce its density in areas of particular blend values, which is why in this screenshot that path going up the hill is free of grass. This tremendously reduces the amount of manual tweaking required to make the grass not do obviously-bad things.

However, the grass doesn’t check for buildings yet, which means if you place a field of grass across the area of a building, the grass will probably stick up through the floor inside the building! Fixing that is next on my to-do list, but it involves a lot of raycasting, so hopefully it doesn’t make grass placement too slow.

In the future I plan to try using this same “grass” system for other kinds of scattered ground objects (fallen leaves, twigs, small rocks, etc).

Toward the back you can see some of the stuff that was being worked on in the previous post (but from a different angle!)

The current puzzle count is 251. That’s pretty good, but on top of the increase from last time, another 25 of the old puzzles were rewritten. So a bunch of stuff has gotten done!

Over the next couple of weeks, I will be hammering on the puzzles even more, as we get the game to a new level of completion and playability.

I was talking to David Hellman about the game the other day, and he was asking when the island was going to become less like a flat plane where you can just walk to any destination from anywhere else.

It’s been on my to-do list for a while to add more topology to the island. That week I had been working on an area full of puzzles, and I wanted the buildings that define this area to have more of a purpose than they currently do; a topological purpose seemed like the right thing.

So I figured I’d add a canyon, and the building containing these puzzles would also serve as a bridge across the canyon.

The natural way to build the canyon in this engine is to shape it out generally with terrain control points, and then make the walls steep by putting mesh objects in there (so that you can’t walk up them, and so that the walls of the canyon won’t look super-distorted the way heightfield terrains usually do when there are steep slopes.)

Here was the first cut at it:

As you can see, there are a lot of weird ripples and curves happening there. This is what signal processing people might call “ringing”. What’s happening is that the Kriging system that we are using for terrain modeling is always trying to reconstruct a smooth function, but what we are trying to build here is a discontinuous function (or else a function so steep that it has a lot of high frequencies in it).

There are a number of ways that one might fix this, but the cheap and hacky one, which would work not only for Kriging but for any other terrain modeling stuff we might stick in in the future, is to add the notion of separating planes that represent discontinuities in the terrain. When evaluating the height of the terrain at any point, we just ignore any control points that are on the other side of the separating plane.

Adding discontinuities has been on my to-do list since last year when Kriging was added to the engine. It was pretty obvious that it would be needed eventually, but in the spirit of getting things done and not rat-holing on technology, I chose to put it off for as long as possible. But it makes sense that if we are starting to add more topology to the island, we are going to want to have more steep places, and so it’s time to add this.

(We’ve already had this steepeness problem before, though: if you look at this plateau that is near the camera in our regular island snapshots, you’ll see that the whole thing is made out of mesh objects, rather than terrain; we did this in order to avoid this very same ringing problem:)

It didn’t take long to add discontinuity entities into the editor; here’s how they are represented:

To start with, I just used a really dumb heuristic to decide when control points should be excluded. When we use this, here’s the result:

There is still a bunch of heightfield terrain sticking out. What’s going on? If we look back at our red discontinuity planes, it is clear that there are some terrain sample points inside the canyon that are getting treated as if they were outside:

What’s going on is that my simple criterion said “if you are on the opposite side of a discontinuity plane from a control point, ignore that control point”. The problem is that I am treating them as infinite planes, so those points that you see, which are in front of one plane, are actually outside a neighboring plane, if you were to extend that plane infinitely. This is no surprise; I knew it would be a problem, but I am always about implementing the simple thing first and seeing what happens. But now I need a heuristic that will fix this problem.

One solution would be to combine the discontinuity objects into sequences of points, and then do some more-heuristicky things that may be easier because we have an ordering between the points.  But I don’t want to resort to that if I don’t have to, because that may make the system less general, and may make it harder to edit.  (Then again it may make it easier to edit, because we have editor support for paths!  But if we want to, we can easily go the other way, from a path down to individual slabs, so it’s better to take the most-primitive thing as input if we are able to.)

I added a check that would make discontinuity planes irrelevant if the sample points are closer to the side planes of these very thin slabs than they are to the face planes.  In theory that would at least help, but upon running this, the scene looked almost exactly the same.

Then upon further investigation I found that I was computing totally wrong planes in the Kriging code; I was transforming the coordinates incorrectly, and so the planes were not in fact where the red slabs in the picture would show them to be.  I fixed this:

Ahhh, much better.  This seems like it will be good enough for now.  There’s still the question of how to seamlessly blend the heightfield terrain into the mesh objects, but that can wait until closer to ship.  For now, it doesn’t need to be pretty; it just needs to let us build things of roughly the right shape.

The heuristic I use for discarding points is still pretty simplistic and may cause problems.  My hope is that most of those problems can be worked around just by changing the planes a bit.  If that turns out not to be true, a little more work will need to be done here!

So, there are 222 puzzles left in the game (I just deleted about 10% of them).

Many more puzzles are going to be added; I would guess that by the time the game is done, there will be over 400. We’ll see; I think this will be a fun statistic to watch, along with those screenshots of the island.

In comparison, Braid has about 76 puzzles in it. So that tells you something about the relative sizes of the games. That said, the puzzles in The Witness are often intended to be bite-sized and quick, so it’s not really a direct comparison. (On the other hand, some of the puzzles are most definitely not bite-sized and quick!)

The past week has been a very productive time in terms of world-building. The island has been shrunk further, with several of the locations that used to be further to the right moved inward. The white-walled location from last time used to be a standalone structure, but is now built into the side of that plateau nearest the camera, and has a new structure next to it. One of the prominent garden areas has been redesigned; you may notice it a bit to the left of the center of the map. On top of all this, a new underground area has been added, and I really like some of the puzzles in there.

When the new area gets finished gameplay-wise I will post some shots of that.

Up next: adding more puzzles!

The most obvious difference from the previous shot is the thicker atmosphere. Ignacio changed the fog model; the fact that we have a bit more fog, combined with the art style we’ve converged on, should help us do high-quality LOD handling for distant objects.

Your bonus picture for today is part of an indoor complex that I am very happy with. This room is all white with a bit of colored glass. Note the lovely lightmapping, even on such slapped-together geometry:

The general requirement was that we need to be able to capture a pre-rendered scene with a really wide viewing angle. At first I was confused about the technical aspects of the problem (hopefully forgivable, since I spend most of my time thinking about gameplay these days, so my tech is a little rusty), so I thought that we might need to have a linear projection in order to solve the specific problem under consideration. So Shannon made a mock-up scene in 3D Studio MAX and we started making prerenders with increasingly wide camera angles, in order to test our special environment mapping. Here are three shots of the same scene with fields-of-view of 90 degrees, 120 degrees, and 150 degrees:

(Click on the images to see actual sizes.)

It’s clear that as the field of view becomes wider, we can see more of the scene at once, laterally. This is good for our application, because we want to see as much as possible! But at the same time, when the angle is wide, objects at the center of the scene appear further away, occupying fewer pixels in the rendered image. This is bad because it means we only have low-resolution imagery for the most important things in the scene! (These screenshots are also badly artifacted because for some reason the avi encoder we were using interlaces the output by default, and we couldn’t find a way to not interlace, without using a different codec! Hint to anyone making a video codec, anywhere in the past or future of this universe or any other: NOBODY LIKES INTERLACING. IT IS UGLY AND CAUSES A LOT OF PROBLEMS. DON’T DO IT. Please do your part to make the world a better place.)

It’s around this time that I realized we didn’t have to use a linear projection, which was good — if we can warp the image however we need to, then we can have the important parts of the scene landing really big in the middle of the texture, using lots of pixels, and squeeze the rest of the scene nonlinearly around the edges of the bitmap, using fewer pixels.

A 180-degree fisheye lens seemed like the right tool to do this. For other reasons that I won’t go into, another nice property of the fisheye lens (as opposed to some arbitrary distortion) is that it is physically plausible — you can mount a fisheye lens onto a physical camera and generate a similar image.

We found some plugins for Mental Ray (a renderer that you can use from Max or Maya) that seemed to do the trick. Here’s the output of one of the plugins we found (this is from Jan Sandström’s JS_fisheye.c, which appears to be a modification of a simple fisheye lens shader in the Mental Ray reference manual):

Unfortunately the field of view in this image we saved is less than 180 degrees, which makes it hard to directly compare with the later images, but just look at the basic character of it for now.

At first I assumed that this shader was implementing actual fisheye lens math, though there were no real comments to go on. Here’s the math used in the stripped-down Mental Ray Reference Manual:

    mi_vector_to_camera(state, &camdir, &state->dir);
    t = state->camera->focal / -camdir.z /
           (state->camera->aperture/2);
    x = t * camdir.x;
    y = t * camdir.y * state->camera->aspect;
    r = x * x + y * y;
    if (r < 1) {
        dir.x = camdir.x * r;
        dir.y = camdir.y * r;
        dir.z = -sqrt(1 - dir.x*dir.x - dir.y*dir.y);
        mi_vector_from_camera(state, &dir, &dir);
        return(mi_trace_eye(result, state, &state->org, &dir));
    }

(JS_fisheye.c has more parameters but is basically the same thing.)

Essentially, this code takes an input texture coordinate and converts it into a view vector in 3D space. In order to use the rendered image from within a shader in the game, I need to be able to invert this function: turn it from a view vector into a 2D texture coordinate. But when I tried to do this, I had all kinds of problems: the math was messy and ugly. I got the inkling that this shader may not be acting as a physical lens really would, but rather, was a 2D image-warping effect that gives the same general impression as a fisheye lens.

(The reason for thinking this is that light behaves bidirectionally; there’s no difference between light moving forward and light moving backward. So if something is simple when light is going in one direction, it ought to be simple in the other direction too. If the equation starts looking a lot more complicated, that is sort-of a violation of the way that the physical universe works, mathematically, and so one starts thinking that something is wrong. That was the intuition I had, anyway.)

Over on this page I found a nice reference for the way that a real fisheye lens bends light. Indeed the equation is very simple (though as it is written on that page, it’s not quite in the proper form for us to use). Based on that, I wrote a new Mental Ray shader that looks like this:

    mi_vector_to_camera(state, &camdir, &state->dir);
    miScalar x = state->raster_x / state->camera->x_resolution * 2 - 1;
    miScalar y = state->raster_y / state->camera->y_resolution * 2 - 1;

    miScalar r2 = x * x + y * y;

    if (r2 < 1) {
        miScalar c = 1 - r2;
        miScalar s = sqrtf(2 - r2);

        dir.x = x * s;
        dir.y = y * s;
        dir.z = -c;  

        mi_vector_from_camera(state, &dir, &dir);
        return mi_trace_eye(result, state, &state->org, &dir);
    }

Superficially it doesn’t look too different from the previous example, but in fact this version refracts light in a physical way and is invertible. Here’s what a render looks like using this lens:

As I mentioned, it’s hard to compare with the earlier shot because the field-of-view is different (sorry about that), but I think it’s evident that the nature of the distortion is fairly different between the two shots. In our version, it feels milder.

Here’s the runtime pixel shader code that inverts it:

    // (xprime, yprime, zprime) is the view vector in the same space where the environment map was rendered.

    float c = abs(zprime);

    video_uv.xy = float2(xprime, yprime);
    float scale = 1 / sqrt(1 + c);
    video_uv.xy *= scale;

    // Now we have video_uv in [-1, 1]; for [0, 1], scale appropriately.

It’s very simple, and the only real math required is a multiplication by a reciprocal square root (very fast for shaders!) So that was pleasant.

Once we had these both hooked up and working, the scene rendered perfectly, and we knew that the effect we wanted to create was achievable. By way of improving it, Ignacio suggested using a cylinder map instead of a fisheye lens, because that is better for this particular shot: we need to render a wide room, and see a lot laterally, but the vertical span is roughly constant and much shorter than the horizontal span. (The fisheye lens is more general, and we can use it for any scene, but to optimize texture resolution for specific cases, we might go to other things like the cylinder map. You can think of the cylinder map as being fisheyed along only one axis.)

Here’s the cylinder map version of the scene:

This is what we are going with for now. Problem solved, job seemingly well-done.

Ignacio put the cylinder mapping code into the same shader as the fisheye lens, and also added a latitude-longitude distortion.

Our work was sped up drastically by the fact that we were able to find JS_fisheye.c free on the internet, as well as information for the way a fisheye lens works. So here’s our attempt to give back a little: our final Mental Ray shader and the .mi file that defines the interface for it:

witness_fisheye.c
witness_fisheye.mi

If you are an experienced graphics programmer asking yourself why the hell didn’t they just use a cube map for this, well, there is a very good answer to that, but it involves the spoiler. You’ll see when the game is released!

The final result is published in Edge issue 223. I am very happy with it. It is probably my favorite interview I have done during my 15 years in the game industry. I thank Edge for the opportunity to do the interview and for their hard work in writing it up.

Because I like the interview so much, I should be happy when other news sites spread the word about it, right? Apparently it is not so. CVG just published online an article about the interview titled

They fill out their article by pasting in somewhat-out-of-context quotes to support this statement (and nothing else). To believe their account, it was a highly anti-Microsoft and pro-Sony interview. [I am not going to link to CVG's article, because I don't want to give them hits; go find it on a search engine if you want to see it.]

CVG’s article is a deceptive, manipulative piece of sensationalist crap meant to drive hits by stoking the argument between Sony fans and Microsoft fans. It misrepresents the content of the interview almost entirely. (Chris Hecker, who was half the interview, doesn’t even appear in their write-up. This alone should make it clear that they have no interest in truthful and accurate communication.)

Chris Hecker: Half the interview! He's even in the foreground in this picture.

For the record, here’s a list of the subjects Chris and I actually talked about:

• Overall trends in indie game development — where, and for how long, will indies be able to find Braid or World of Goo-sized audiences for their games?
• How it’s easier to make indie games, on the technical implementation side, than ever before.
• Small indie games versus big indie games — punk-rock-esque one-offs vs deeply exploring game design subjects.
• Indies ought to step up and produce higher-quality work rather than so many bad games and a few good ones.
• Braid is structurally a traditional game in many ways.
• The idea of game design as a stream of focused communication rather than a set of rules.
• Even the highest-selling games right now essentially sell to a niche audience.
• Innovation in mainstream games and how it helps drive the industry.
• When is an indie game really finished — how much work should you put in?
• Problems with the way video game education is set up right now.
• Most people who use the phrase “game mechanic” don’t seem to know what it means.
• How to see your game from the perspective of the player.

In the middle of all this, there is one bit about the way Microsoft and Sony, respectively, handle their portfolio strategies with regard to indie games. There are positives and negatives mentioned regarding each party. Sony does come out a bit ahead, but our discussion was much more even-handed than the confrontational CVG headline would have you believe.

From CVG's write-up, you would barely even know that I talked about my company's upcoming game The Witness.

Is it the case that I said the things that CVG quotes? Yes! Are they an honest representation of the content of the interview? No! They are a dishonest and manipulative way of driving hits to CVG’s web site.

Chris recently wrote about this kind of treatment at the hands of journalists, so rather than repeating him, I encourage you to go read that article. I do want to point out, though, that this is not a victimless crime.

When this kind of hack-job is written, it hurts indie developers: it harms our relationship with our business partners like Microsoft or Sony when they think that we called an interview just to trash them in public (or when they know we didn’t really do that, but they have to deal with the PR fallout of the public thinking we did, which is just as bad.) It harms my reputation with players because they think I called an interview just to make a big deal out of bashing Microsoft and generally being negative (which I did not do!) It does an injustice to people who want to read about thoughtful discussions in video games; it does this not only by failing to point out the thoughtful parts of the interview, but by injecting extra noise into the sphere of discussion, making it harder for everyone to find anything thoughtful.

But perhaps worst of all, CVG is crapping on their fellow journalists at Edge by depicting Edge’s hard work as tabloidesque junk.

Damage is clearly done, and CVG are doing this damage just so they can make a little bit of money off a few more web hits.

I encourage everyone out there to stop reading this kind of stuff. It’s just misinformation; it’s worse than worthless.

One interesting change, compared to the previous shots: you’ll notice that the lake that used to be in the center of the island, with the windmill on the edge, has moved over toward the left side and become a bay.

This is one of those interesting area where design and technical constraints interact heavily. We wanted the water of that lake to be reflective, but the way 3D graphics works, you basically need to render the entire scene an extra time in order to make a reflective surface. For outdoor scenes this is very expensive. Now, we also wanted some of the water out at the edge of the island to be reflective. If the inland lake and the water near the beach were at different elevations, this would mean we need to render the entire scene twice, all the time, to generate water reflections, before rendering the main scene. This can be optimized somewhat, but in an open-world game, there is only so much you can do.

The workaround to this was just to keep the inland lake at sea level, so that we can use a single rendering pass to generate all water reflections. Technically this worked fine, but design-wise, it meant that we had to have this big spot in the middle of the island that was at sea level, while also having everything at the edge of the island being at sea level (by definition). This gives the island a doughnut kind of shape, and given that the island is so small, this ended up severely restricting how many elevation changes can be introduced — the island was being artificially influenced to be flatter than it ought to be, because of this constraint of being sea-level in the middle.

So I moved that lake off to the side and connected it to the ocean. This involved moving a couple of other areas as well. The gameplay is preserved, and the new position is actually better for certain gameplay purposes. The island is still pretty flat, but I think you’ll see it getting less flat over time, now.

I had been viewing a bunch of online portfolios of 3D modelers (and lamenting the quality of these portfolios) with an eye to hiring someone. Sometimes these portfolios are shown in steps: here’s the wireframe, here’s the filled-in geometry, here it is with lighting, and here is the final thing after it’s been textured. Casey pointed out something interesting about these that I knew was true as soon as he said it: most of these portfolios look worse than they need to because the texturing ruins them. When you run a radiosity algorithm on a scene full of unpainted geometry, it has an elegant, sophisticated, scent-of-the-real kind of thing going on; but it is very easy to spoil this sophistication through texturing. And, it seems, in video games we have developed a tradition of “bad” texturing, whatever that means: unsubtle, unrefined, or just plain ugly.

We’re on a low budget here, being an indie game; so not only was it an open question how to solve the Ugly Texturing problem, but we also needed to solve it with relatively little manpower.

I knew that, ultimately, we could just fall back on untextured geometry with lighting, if we had good precomputed lighting. That’s why we spent so much time and effort putting together our lighting system (as has been written about many times in this blog). The question was, would we stick to vertex colors on most objects, or would we use some kind of texturing?

Early on, we identified a texturing style that would work for human-sized objects and would gel with the gameplay of The Witness. The style prioritizes color, and tries not to break up the geometry of the object by imposing conflicting details through the texture. We did some objects like a 1-person boat in this style in order to refine it. [I'd like to include a screenshot of the boat here to illustrate it, but the boat is messed up right now, so I'll just have to be vague.]

Because that style tries to use the texture map to accentuate the object, the contents of the texture must change with the local neighborhood of the object, which means you can’t use tiling textures very much. This means it would be very expensive to texture something like a big building, since you’d need unique texels everywhere.

Thankfully, the Western game development world has a tradition of developers being very open with their techniques and processes, and communicating with other developers very honestly about what is effective. (This is not something that one finds elsewhere in the world, and we should work hard to keep that spirit alive because it benefits everyone; but that’s a totally separate rant.) Here’s one good example of this kind of information exchange: at the GDC this year, some guys from Naughty Dog gave a talk on the art direction for Uncharted 2 (pdf link to their slides here.) Somewhere around page 220 of this very informative slideshow, they start talking about the blend shaders that they use in order to give more character to buildings and other geometry while still using tiling textures. The idea is that you interpolate between two (or more) textures with a blend function that fits the shape of your building nicely. You put some smarts into the interpolation so that it produces something that is visually interesting, rather than the blurry mess you would get via naive interpolation.

For The Witness it seemed like a good idea to use this kind of technique, but in a minimalist way, to create patterns that accentuate the shapes of buildings, and help tie them together visually, without stumbling into the realm of Ugly Texturing.

(There are a lot of reasons for the minimalism; one reason, though, is to provide clarity about what you can interact with, versus what is inert. Interactive things are colored in a way that makes them stand out in an obvious way. Think of it as something like what Mirror’s Edge did, but in a different way.)

We’ve started experimenting with these blend shaders; here are some images:

The upper-left shows a scene before we implemented the blend shader for buildings. In the upper-right, we are using a blend shader that blends the original texture with an alternate one that is a bit darker and rougher (but the difference is a bit subtle, by game standards at least). Below these images are the same scenes with precomputed lighting mixed in.

It is pretty clear to me that the lighting is actually the biggest ingredient in tying the scene together, but the blend shader is, I think, a good extra kick. It helps the area feel like there is a little bit of life and history to it (for example, the lighter-colored paths near the puzzle panels in the foreground, which imply that people have been walking that way). What you see here is not of shipping quality, but I see how we can reach shipping quality by iterating on this style and putting production-level effort into the assets.

Here’s a close-up of what the stairway looks like:

As you can see, the texture maps have some grain to them; from far away, the grain melts away and the structure is mostly a solid color, accented by lighting.

These experiments are still early yet but I find them very promising. The plan is that we will continue to hone the style and end up with something very special.

This talk is called Games and the Human Condition; it was given at Rice University on September 27, 2010. It’s about “best practices” of modern game design that I find unsettling, and the way in which “social games” and “gamification” are destructive.

A full video is available from this page at Rice University.

Here’s a link containing just the slides (ppt format).

Here’s a long and wordy abstract:

Video games have evolved tremendously over the past few decades; they’re much more entertaining than they used to be. That is not by accident; we, the community of game designers, have been continuously refining our techniques. The most common way we do this is by testing out our games on you, the players, and optimizing for the “best” result (where “best” is defined by us). As this process is ongoing, what kind of relationship exists between the designer and the player? Is it artist/audience, experimenter/subject, entrepreneur/customer, or tycoon/resource? Invariably it’s some admixture of these things, the particular ratios for a given game being chosen by its designers (usually without awareness that a decision is being made). Today, due to the way the Internet is widely used, and because game designers are becoming more serious about certain aspects of their craft, the iteration time of this game design optimization process is shorter than ever before: designers can observe their players much more thoroughly, and more quickly, than they ever have in the past. At some point a quantitative change becomes a qualitative one: the result of all this competency may be heavily destructive. Some aspects of the current notion of “good game design” may in fact be very bad, or at least indefensible, from an ethical standpoint. Today’s “better” video games spend a great deal of effort to undermine defenses that took you tens of millennia to evolve. They tend to be successful at this. As designers keep evolving their craft and gain greater analytical power, what will happen?

I’ve seen mixed reactions to these mazes and to the gameplay shown in the video. Some people seem to have jumped to the conclusion that the game is going to be about solving simple maze puzzles, and thus not be particularly exciting.

If that thought crosses your mind, all I can ask is to have faith. I wouldn’t make a game about solving a series of rote puzzles. The footage came from the beginning of the game because we don’t want to spoil anything major. But even this beginning-of-the-game stuff has more to it than meets the eye.

It’s like Braid, in a way. Back before Braid was released, I never made a trailer video for it, because I couldn’t come up with a way to show in video form what the game was about. You can show some rewinding happening, or other funky time stuff, sure, but what is cool about Braid is not the mechanics of those things or how they look on the screen. What is cool about Braid is what happens in the player’s mind during the puzzle-solving process. The time stuff is the method by which that is implemented, but the rules of time manipulation do not contain the magic. Watching a video, it’s nearly impossible to see the magic. You have to play the game to see the magic.

The Witness is like that too. The blue mazes are a major aspect of the game, but they are not the point. The point is the magic that happens in the player’s mind when he understands the subtle things that the mazes are saying — because the mazes aren’t just puzzles, they are lines of communication that aggregate, become more complex and eventually say surprising things. This can’t be seen in a video; you have to play it to have the experience. The magic is in the player’s head, not on the screen.

The only two guys who have played The Witness to completion have said it will be better than Braid when it is done. I am certainly not going to jump up and down and say “hey this game is better than Braid”, or even claim that Braid is good. But I just want to put that out there as reassurance to those of you who are worried about that gameplay video.

I have a file with about 80 game ideas in it, most of which would probably make really good games. But I can only make one game at a time right now, so I had to pick one. I picked The Witness, even though it more difficult and expensive to make than most of the other ideas, because I was more excited about the core concept of The Witness than about the other 79 ideas. This is a highly ambitious game that stretches my ability as a game designer to its limit.

So if you don’t yet see why the game is ambitious or exciting, please have patience! We’ll be releasing more information as the game comes closer to completion…

And whatever you think about what you’ve seen so far, I’d like to thank you for your interest in the game!

Using the same engine makes it very easy to keep the global illumination in sync with the rest of the game. If we add new shaders or materials, change the representation of the geometry or alter the world, keeping the global illumination up to date requires little o no effort. Another interesting benefit is that optimizations to the rendering engine also benefit the global illumination pre-computation. However, as we will see later, in practice each application stresses the system in different ways and end up limited by different bottlenecks, what improves the rendering time does not necessary improve the pre-computation and vice-versa.

Projection Methods

I tried several different approaches to render the scene and estimate the incoming radiance. The most common approach is to render hemicubes; that requires rendering the scene to 5 different viewports. Several sources suggested the use of spherical or parabolic projections, but in my experience these approaches produced terrible results due to the non linear projection. Without very high tessellation rates the scene would become unacceptably distorted, creating gaps between objects or making parallel surfaces self-intersect.

In GPU Gems 2, Coombe et al. suggested to project the z value independently of (x,y) coordinates using an orthographic projection in order to maintain a correct depth ordering. As seen in the next picture, that removed the most obvious artifacts; note how the staircase inside the tower is not visible anymore:

However, while this works OK from some points of view, the distortion becomes unacceptable when the camera is close to a surface, which is always our case. So, in practice these approaches are not very useful. Note how the outdoors are visible when sampling the lighting from the interior of the tower:

Other implementations use a single plane projection. This kind of projection does not capture the whole hemisphere, but by using a wide field of view it’s possible to capture about 80% of it. I haven’t actually tried this approach, but to get an idea of what the results would look like, I modified the hemicube integrator to ignore 20% of the samples near the horizon and found out that this introduced artifacts that were unacceptable for high-quality results.

Pascal Gautron estimates that on typical scenes this approach introduced a 18% RMS error, but suggests various tricks to reduce it by re-balancing the solid angle weights, or extrapolating the border samples and filling the holes. The latter method reduces the error down to a mere 6%.

However, these improvements do not eliminate banding artifacts caused when an intense light source visible from one location is suddenly not visible anymore in the next when it falls out of the projection frustum. To make things worse, the projection lacks rotational invariance, which enhances the problem when hemicubes are rendered with random rotations. So, while in many cases the results look alright, the few cases that do not render the method impractical.

Another problem is that the wide frustum is 4 times larger than the equivalent hemicube and easily intersects with the surrounding geometry; as I’ll discuss next, this is one of the main problems that I had to deal with.

In the end I simply stayed with the traditional hemicubes for simplicity. I think it would have been possible to do somewhat better using a fancy tetrahedral or pyramidal projection with 3 or 4 planes, but that didn’t seem worth the trouble.

Hemicube Overlaps

As I just mentioned, one of the main problems that I had to solve was to handle overlaps between the hemicubes and the geometry. These overlaps are unavoidable unless you impose some severe modeling restrictions that are not desirable for production.

When the geometry is well formed and does not have self-intersections, these overlaps can sometimes be avoided with centroid sampling. Without centroid sampling the location where the hemicube is rendered from can easily end up being outside of the polygon that contains the sample and thus inside another volume.

With centroid sampling the origin of the hemicube is now always inside the surface. However, it can still be very close to the surface boundary and have overlaps with nearby surfaces. A simple solution is to move the near plane closer to the camera until the overlap dissapears, but how close? The distance to the nearest boundary can be determined analytically, but there’s a limit in how much the near plane distance can be reduced without introducing depth sampling artifacts. An interesting solution for this problem is to render multiple hemicubes at the same location, each hemicube enclosing the previous one with its far plane at the near plane of the next.

That would solve most the problems when the geometry is watertight and centroid sampling is used, but in many cases our assets are not always well conditioned. So, I had to find a more general solution.

I tried all sort of heuristics, to render backfaces in black, to fill them with the average color, to extrapolate the nearest colors, etc. None of these solutions worked well in all cases. In the end, something that produced satisfactory results was to render the backfaces with a special tag to distinguish them from front facing faces when integrating the hemicubes. Whenever a hemicube had too many tagged texels it would be discarded entirely, and instead its output radiance would be estimated at that location interpolating or extrapolating it from the nearest valid samples.

The extrapolation is not always correct, and the detection of invalid hemicubes is not 100% accurate, but so far this is the method that I’m happiest with. In the next article I’ll explain how this extrapolation is done in more detail.

Shadow Rendering

While the goal was to use the same rendering engine, using its shadow system ‘as is’ would have been overkill. We are using cascade shadow maps, which are updated on every frame, and to maximize the shadow map resolution the shadow maps are tightly aligned to the view frustum slices. When rendering the hemicubes, shadow map resolution is not that important, and re-rendering the shadows for each hemicube face is unnecessarily expensive. To keep things simple I still used cascade shadow maps, but instead of rendering them for each view, I only rendered them once for each object positioning the cascades around the center of the object.

Rasterization

Now that I’ve explained how to capture the irradiance of the scene at any point, it’s necessary to sample it for every lightmap texel. To do that I simply rasterize the geometry in the lightmap UV space using a conservative rasterizer and render one hemicube at every fragment whose coverage is above certain threshold.

One nice thing of using our own rasterizer is that I can evaluate the exact coverage analytically. Since the parameterization does not have overlaps, the coverage is just the area of the intersection of the texel square with the polygon being rasterized. Similarly, the centroid is the center of mass of the clipped polygon.

This approach gives us a position and a direction to render the hemicube from, but I still need to determine the orientation of the hemicube. Initially, I simply chose an arbitrary orientation using the traditional/standard method, but that resulted in banding artifacts:

The source of the problem is just the limited hemicube resolution, but it’s surprisingly hard to get rid of it entirely by only increasing the hemicube size. A better solution is to simply use random orientations:

Note that I am trading banding artifacts by noise, but the latter is much easier to get rid of by using some smoothing.

Integration

Once the hemicubes have been rendered it’s necessary to integrate the sampled irradiance to compute the output radiance. The integral of the irradiance hemicube is simply a weighted sum of the samples, where the weights are the cosine-weighted areas of the hemicube texels projected on the hemisphere, that is, the cosine-weighted solid angles of the texels. These weights are constant, so they can be precomputed in advance.

In the article by Hugo Helias that I referenced in the introduction the same approach is used and this weight table is called the “The Multiplier Map”. Beware of that part of the article, though, because there’s a bug in the formulas. The subtended solid angle of the texel is not “the cosine of the angle between the direction the camera is facing in, and the line from the camera to the pixel”:

That term needs to be divided by the squared distance between the location of the camera x and the center of the texel y:

See formula 25 in the Global Illumination Compendium for more details.

Also, since this is a precomputation it made sense to evaluate the solid angle exactly. That can be derived using Girard’s formula or directly using the handy formula in Manne Ohrstrom’s thesis. Doing so improved the invariance of the integral with respect to rotations of the hemicube and that reduced noise significantly.

In the end, the integration is a fairly simple operation. It could be done very efficiently using compute shaders and I might still do that in the future, but for prototyping purposes it was much easier to perform the integration in the CPU. Unfortunately, that required transferring the data across the bus and, if not done properly, it severely reduced performance.

Asynchronous Memory Transfers

Contrary to popular belief, Direct3D 9 provides a mechanism to perform memory copies asynchronously. In Direct3D 10 and OpenGL this is done through staging resources or pixel buffer objects; in Direct3D 9 something similar can be achieved with offscreen plain surfaces. The problem is that this is not very well documented, it’s very easy to fall out of the fast path, and the conditions in which that happens are slightly different for each IHV.

In the first place, you have to create two render targets and offscreen plain surfaces:

dev->CreateRenderTarget(..., &render_target_0);
dev->CreateRenderTarget(..., &render_target_1);
 
dev->CreateOffscreenPlainSurface(..., &offscreen_surface_0);
dev->CreateOffscreenPlainSurface(..., &offscreen_surface_1);

If you have more than two targets the copies become synchronous in some systems, but two is sufficient to generate enough work to do in parallel with the memory transfers.

The most important observation is that the only device method that appears to be asynchronous is GetRenderTargetData. After calling this method, any other device method blocks until the copy is complete. So, instead of overlapping the memory transfer with the rendering code, you have to find some CPU work to do simultaneously. In my case, I just do the integration of one of the render targets while copying the other:

while hemicubes left {
 
    [...] // Render hemicube to render_target_0
 
    // Initiate asynchronous copy to system memory.
    dev->GetRenderTargetData(render_target_0, offscreen_surface_0);
 
    // Lock offscreen_surface_1
    // at this point the corresponding async copy has finished already.
    offscreen_surface_1->LockRect(..., D3DLOCK_READONLY);
 
    [...] // Integrate hemicube.
 
    offscreen_surface_1->UnlockRect();
 
    swap(render_target_0, render_target_1);
    swap(offscreen_surface_0, offscreen_surface_1);
}

Another problem is that the buffer needs to be sufficiently big to reach transfer rates that saturate the available bandwidth. In order to achieve that I batch multiple hemicubes in a single texture atlas as seen in the following picture:

In practice I do not render the hemicubes using the depicted cross layout, but instead arrange the faces tightly in a 3×1 rectangle that does not waste any texture space and maximizes bandwidth. However, the cross layout was useful to visualize the output for debugging purposes.

Performance and Conclusions

The performance of our global illumination solution is somewhat disappointing. While we do most of the work on the GPU we are not really making a very effective use of it. The main GPU bottleneck is in the fixed function geometry pipeline; we are basically rendering a large number of tiny triangles on a small render target.

But that’s not the main problem. Traversing the scene in the CPU and submitting commands to render it actually takes more time than it takes the GPU to process the commands and render the scene, so the GPU is actually underutilized.

Our engine was designed for quick prototyping, to make it very easy to draw stuff with different materials, but not to do so in the most efficient way possible. These inefficiencies became a lot more pronounced in the lightmap baker.

On the bright side, as we optimize the CPU side of the engine we expect performance to improve. Modern GPUs also have functionality that make it possible to render to multiple hemicube faces at once. We expect that will also reduce the CPU work significantly.

While it was easy to get our global illumination solution up and running by reusing our rendering engine, bringing it to a level of sufficient quality and performance took a considerable amount of time. In the next installment I’ll describe some of the techniques and optimizations that I implemented to fix some of the remaining artifacts and improve performance.

Here’s an example. The following tree looked OK close to the camera:

but as we moved away, the leafs started to fade and thin out:

until they almost disappeared:

I had never encountered this problem before, neither had I heard much about it, but after a few google searches I found out that artists are often frustrated by it and usually work around the issue using various hacks. These are some of the proposed solutions that are usually suggested:

• Manually adjusting contrast and sharpening the alpha channel of each mipmap in Photoshop.
• Scaling the alpha in the shader based on distance or on an lod factor estimated using texture gradients.
• Limiting the number of mipmaps, so that the lowest ones aren’t used by the game.

These solutions may work around the problem in one way or another, but none of them is entirely correct and in some cases add a significant overhead.

In order to address the problem it’s important to understand why the geometry fades out in the distance. That is because when computing alpha mipmaps using the standard algorithms, each mipmap has a different alpha test coverage. That is, the proportion of pixels that pass the alpha test changes, in most cases going down and causing the texture to become more transparent.

A simple solution to the problem is to find a scale factor that preserves the original alpha test coverage as best as possible. We define the coverage of the first mipmap as follows:

coverage = Sum(a_i > A_r) / N

where A_r is the alpha test value used in your application, a_i are the alpha values of the mipmap, and N is the number of texels in the mipmap. Then, for the following mipmaps you want to find a scale factor that causes the resulting coverage to stay the same:

Sum(scale * a_i > A_r) / N == coverage

However, finding this scale directly is tricky because it’s a discrete problem, in general, there’s no exact solution, and the range of scale is unbounded. Instead, what you can do is to find a new alpha reference value a_r that produces the desired coverage:

Sum(a_i > a_r) / N = coverage

This is much easier to solve, because a_r is bounded between 0 and 1. So, it’s possible to use a simple bisection search to find the best solution. Once you know a_r the scale is simply:

scale = A_r / a_r

An implementation of this algorithm is publicly available in NVTT. The relevant code can be found in the following methods of the FloatImage class:

float FloatImage::alphaTestCoverage(float alphaRef, int alphaChannel) const;
void FloatImage::scaleAlphaToCoverage(float desiredCoverage, float alphaRef, int alphaChannel);


And here’s a simple example of how this feature can be used through NVTT’s public API:

// Output first mipmap.
context.compress(image, compressionOptions, outputOptions);
 
// Estimate original coverage.
const float coverage = image.alphaTestCoverage(A_r);
 
// Build mipmaps and scale alpha to preserve original coverage.
while (image.buildNextMipmap(nvtt::MipmapFilter_Kaiser))
{
    image.scaleAlphaToCoverage(coverage, A_r);
    context.compress(tmpImage, compressionOptions, outputOptions);
}

As seen in the following screenshot, this solves the problem entirely:

even when the trees are far away from the camera:

Note that this problem does not only show up when using alpha testing, but also when using alpha to coverage (as in these screenshots) or alpha blending in general. In those cases you don’t have a specific alpha reference value, but this algorithm still works fine if you choose a value that is close to 1, since essentially what you want is to preserve the percentage of texels that are nearly opaque.

For all three days of PAX 2010, The Witness was publicly playable by anyone who came by the booth.  However, it was unmarked and unattended, so it was easy to miss (as many people did; but then, plenty of people played it, too).

Here’s a close-up, with the opening scene of the game, as it silently waits for someone to sit down and play:

I had several reasons for wanting to show the game this way.  Firstly: At a show full of companies trying to capture your attention and sell you things, I wanted to do something that is subtle, and a surprise — if you notice it, and decide to investigate, you find something unexpected.

Also, I wanted people to be able to play the game for as long as they want, not feeling pressured to stop playing because of a huge line of antsy people waiting behind them.

Before PAX, only two people had ever played the game; those people were game developers, and the versions they played were very rudimentary.  This was the first time the game had ever been played by general players, and I wanted to see how people would react to it, as honestly as possible.  I didn’t want to give them any reason to think that this was a good game, or was by any designer they might have heard of.  So I just left the game running, and would occasionally peek at it from afar to see who was playing and how they were doing.

PAX is a huge show, with tons of stuff to see everywhere; if the game could hold peoples’ attention under those conditions, purely on its own merits rather than by hype or pedigree, then I would know that it was really working.

Happily, the game worked very well — despite the fact that the visuals are still in an early prototype stage.  People tended to play for a long time, and they liked what they played.  Here’s a picture from Sunday; these guys played for a long time, perhaps an hour and a half:

Some game designers did recognize the game; here’s Dylan Fitterer (of AudioSurf) playing:

Stephen Totilo of Kotaku recognized the game early, on Friday, and found me camping out on the other side of the booth.  Thankfully, though, he didn’t spill the beans.

I am very happy with how this playtest session went.  I am glad I was able to bring a game directly to players, as a surprise.

The common wisdom among game designers is that the first time you put your game in the hands of players, it totally falls apart, and nothing works the way you expected.  Thankfully, that didn’t happen here.  The game does work as expected, despite being kind of crazy and experimental, and people enjoy playing it.

All that’s left now is just to do all the hard work required to finish the game!

Update: Stephen Totilo has posted his impressions, along with a gameplay video, at Kotaku.

A few of the many, many ideas we looked at:

Here’s David, looking like he’s at the top of his game:

We’ll be working on that ugly green area soon…

Today I took a step in that direction by moving some areas around and compacting the island, changing its shape a bit:

Here’s a shot from the other side of the island, where you can see that there is plenty of empty space left:

So-me of this will be filled with new gameplay, some will be removed or reshaped, etc, depending on how things evolve.

Here’s a close-up of a new area; there’s something going on there with a lot of wires, and stuff:

A movie can give you a satisfying experience in 2 hours.

A painting or a sculpture can give you a satisfying experience in 10 minutes.

A song can give you a satisfying experience in 3 minutes.

What is it about certain kinds of linear-experience games that makes players feel they need to play them for hours upon hours in order to accrue a satisfying experience? If it really takes that long, doesn’t that mean the medium is somehow deficient in important ways?

Gamers seem to praise games for being addicting, but doesn’t that feel a bit like Stockholm syndrome? If you spend 20 hours playing a game, but the good parts could have been condensed into 3, then didn’t you just waste 17 hours?

If you waste 17 hours a month for the rest of your life, what is the cost of that, socially, quality-of-life-wise, economically, or however else you want to measure?

I am not saying that all games should be short. Certainly there’s a place for long-form games, just as there’s a place for long-form TV shows like Deadwood or Lost. But if we lived in a world where all cinema had to be the length of Lost, and films like Inception were criticized by cinema fans, and given lower scores in reviews and such, for being too short… wouldn’t that be perverse? Certainly the people in that world would be ignoring a big chunk of the potential of that medium, for arbitrary and weird reasons.

This post is part of an industry-wide commentary by indie developers on the subject of short games. Other participants:

Ron Carmel of 2DBoy

Chris DeLeon

Dave Gilbert of Wadjet Eye Games

Matt Gilgenbach of 24 Caret Games

Michael Todd
Eitan Glinert of Fire Hose Games

Cliff Harris of Positech Games

Chris Hecker of Spy Party

Scott Macmillan of Macguffin Games

Noel Llopis

Peter Jones of Retro Affect

Lau Korsgaard

Martin Pichlmair of Broken Rules

Greg Wohlwend of Intution Games

Jeffrey Rosen of Wolfire

Steve Swink

The Chaparral High School Alumni Theatre’s production of Braid: A Movement Piece during the summer of 2010. Using a unique combination of content movement and contact improvisation dance techniques, Braid follows the story of young scientist Tim on his quest to find a Princess.

We are starting to tinker around with the sky (this one is way too dark). Also, moving around a bunch of locations and adding some new ones.

I’m doing a couple of weeks’ worth of serious map organization / reorganization in order to get the game to a point where we can playtest it at pretty much any time, and whoever sits down to play will have a reasonable experience. The game was in this kind of shape back in January, but we had to do a lot of work on game internals in order to get things where they needed to be, interactivity-wise, for the mid-stage of development. Now that most of that stuff is done, it’s time to put the world/content back in playable shape; from there, new stuff can be added in a way that continuously fills out the world. Once I feel like the world is “full enough” we can leave the preproduction phase and start going into production (which is where we start caring about what the graphics look like, start seriously doing voice recordings, etc).

One area I’ve been working on for the past few days is the start of the game. It is meant to be a semi-guided experience that clues the player into what the game is about. (If you remember the beginning of Braid, it’s something like that.) You may have seen the interior of the house (the white building on the left) in earlier blog posts:

The area just outside the house is a constrained path, but very soon the player exits this area and can freely roam an open world.

Most open-world games want to impress you with the hugeness of the world, and the corollary to that is that you spend a lot of time walking / riding / driving between places. The Witness takes the opposite philosophy: it wants the world to be as high-density as possible, just packed full of interesting things. As the island develops, we will be placing areas as close to each other as we can, within constraints. (Braid had a similar high-density philosophy — though it manifested in a different way — and many people appreciated that).

If I can be said to have an agenda in game design, it has something to do with mind-expansion.  Because of that, I very much appreciate it when other designers make games with that kind of intention.  Games that are truly mind-expanding are very rare and very difficult to make, but this is one of them.

Keep an eye out for this game, and when it comes out, play it.  Since it will be a while before release, so there’s no demo available.

Here’s an introduction to the game.

Here’s an interview Marc did with Gamasutra with further details.

To keep up with news on the game, you can subscribe to Marc’s blog.

The Witness is an exploration-puzzle game, to be released on multiple platforms — whatever makes sense in late 2011, when the game will hopefully be finished.

The game is being developed by a small, independent team, a mix of full-timers and contractors. Here’s a list:

Jonathan Blow
Jonathan is a designer-programmer whose goal is to make games that are mind-expanding in ways special to this medium. He is best known for the well-received game Braid. He is also a partner in the Indie Fund, an initiative to help creative new developers grow stronger while remaining independent. He speaks frequently at conferences and universities on the advancement of game design as an art form.

Ignacio Castaño
Ignacio started programming at an early age and became obsessed with graphics. That led him to work at several game companies including Crytek, Relic and Oddworld. More recently he worked at NVIDIA, where he contributed to the design of modern graphics hardware, APIs, and programming models. Today he’s the father of a 3 year old boy and embraces all aspects of human experience.

Shannon Galvin
Shannon Galvin has worked on games for over 16 years, having started on a game called ‘SimTown’ and then adding pretty pictures to many others…including SimCity 2000, Sims 2 and Spore. When not doing art, he often wanders around games looking at environments and using cheat codes when monsters try to interrupt his art appreciation. Oh, and he’s got a ceramic kiln in his living room. Those things can vaporize metal! (Caution: Kilns should not be used to vaporize metal).

Andrew Hynek
Andrew Hynek has been a game programmer since the early PS2 days. Having worked at small and large studios on a variety of genres, he is trying out the indie / freelancing life at his new company, Atrophy Studios.

Andrew Lackey
Andrew loves sound, nails on chalk board and all. As a gamer and indie music/film/game fan, he’s thrilled to connect with the team on the Witness. Music and sound have been a part of his life since day one, but he got his professional start in Los Angeles on feature films. Go!, The Matrix Trilogy, 8 Mile, and Treasure Planet were among the 30+ films he worked on as a sound editor and foley supervisor. In 2004, he defected to San Francisco and shortly thereafter… the game industry. After a brief stint at Skywalker Sound, he went to work for Electronic Arts as a senior sound designer. He worked on several titles including the creature voices and sound design for Spore and Dead Space…although in decidedly different veins. In late 2008, he went indie and started Wabi Sabi Sound. It’s been a great ride. He’s contributed to BioShock 2, Left 4 Dead 2, Dante’s Inferno, and currently The Witness. The nonconforming push of the envelope is a space he’s always thrived in. The Witness challenges him to rethink the fundamentals of a craft he’s been doing for over 10 years…. he loves that!

Nicholas “Indy” Ray
Nicholas has been programming professionally for 8 years. He first worked as a game developer at the U.S. Army-funded “America’s Army” video game project. He has wide programming experience, ranging from compiler design to animation, and spends much of his time time trying to come up with unique solutions to problems, while stubbornly holding onto the conviction that byte-level data transformation is what programming is all about.

When not programming, he enjoys rock climbing in the summer, various snow sports in the winter, getting lost, and most anything that involves making a fool of himself.

Eric Urquhart
At the age of 7 when Eric won his local community dental art contest by designing a fluffy rabbit with a big shiny tooth. It was then when he knew he could be on to something. So he started practicing painting on his bedroom walls. That wasn’t a big hit with the parents but did serve as good experience for his future as a mural artist. Eric quickly became sought after for his mural work, being flown around the nation to add his personal touch to structures all around the country. Six years ago he set down his analog brush and paints for a Wacom tablet and Photoshop. Since then he has been unleashing his artistic talent in feature film, television and the gaming industries. Past companies and clients Eric has worked for and with include Sony Pictures Imageworks, Warner Bros., Fuel and Brand New School.

It may not look tremendously different from the last one, aside from the translucent water and added objects at the near corner.  However, a number of locations have been worked on in the past few weeks, which happen to be either far across the map or underground, so I figured I’d post some highlights.

Keep in mind that all of this is placeholder graphics; because we consider gameplay to be very important, we are planning out the gameplay carefully before we go into full production.

This area is an old ruin carved from stone; as the world gets developed, the area around it will become more deserty.  In a later blog update I’ll show this area more extensively:

Here’s  a cave at a junction of underground tunnels, lit by colored lights so that you know which tunnel is which:

A cathedral, built on a hill; its walls are old and crumbling, but it’s been well-cared-for:

At the edge of the forest, near a bridge across the river, someone has been cutting down trees.  What was going on here?  Why did they just leave these logs laying around?

We’ve also been working on rendering for grass, bushes, and trees.  Here’s a shot containing some of our new grass:

Trees and bushes used to cast overly-dark, jagged shadows, because the shadow system didn’t know how to look at their texture maps to figure out transparency.  That’s been fixed, and now shadows cast by leaves are the shapes of the leaves:

Much of the other work that I personally have been putting into the game is about the user interface for the puzzles.  As I mentioned, gameplay is the utmost concern; this being a puzzle game, you spend a long time solving puzzles, so it’s important to make that feel good, even at this early stage.  There is still plenty of work to do there, but a very important aspect of the user interface has improved drastically this week, and I am very happy with the results.

More news as it comes!

]]> http://the-witness.net/news/2010/07/location-development/feed/ 10 http://the-witness.net/news/2010/07/the-indie-fund-submissions-process-is-now-open/?utm_source=rss&utm_medium=rss&utm_campaign=the-indie-fund-submissions-process-is-now-open http://the-witness.net/news/2010/07/the-indie-fund-submissions-process-is-now-open/#comments Wed, 07 Jul 2010 03:34:09 +0000 Jonathan Blow http://the-witness.net/news/?p=375 … and the web site is updated.  See it here: http://indie-fund.com/

Earlier this year, some friends and I, all successful independent game developers, put together the Indie Fund in order to help support newer folks with good ideas.  All the hoops involved in setting up such an organization have been passed through, and we are now open for general submissions.  If you are an independent developer who needs money for a project-in-progress, go check out the Apply page to see what kinds of games we are looking for.

Thanks, and good luck!

]]> http://the-witness.net/news/2010/07/the-indie-fund-submissions-process-is-now-open/feed/ 2 http://the-witness.net/news/2010/06/island-snapshot/?utm_source=rss&utm_medium=rss&utm_campaign=island-snapshot http://the-witness.net/news/2010/06/island-snapshot/#comments Tue, 01 Jun 2010 03:47:47 +0000 Jonathan Blow http://the-witness.net/news/?p=370 The addition of the recently-discussed terrain editing code means that we can start giving the island more shape and character, in a natural way.  This is early yet…

]]> http://the-witness.net/news/2010/06/island-snapshot/feed/ 5 http://the-witness.net/news/2010/05/kriging-is-cool/?utm_source=rss&utm_medium=rss&utm_campaign=kriging-is-cool http://the-witness.net/news/2010/05/kriging-is-cool/#comments Mon, 24 May 2010 06:46:10 +0000 Jonathan Blow http://the-witness.net/news/?p=345 In The Witness we decided that it’s a good idea to have a heightfield terrain making up the bulk of the island surface.  The advantages of this are in rendering performance and in ease of texturing (it is relatively easy, with one continuously-parameterized terrain block, to get rid of seams; if your terrain is a bunch of otherwise-unrelated meshes, then what do you do?) When we made this decision it seemed clear to me that the biggest disadvantage, for this game at least, was authoring the terrain.  There are typically a few different ways that large heightfields are built:

1. Use an external terrain editing tool and then write an importer for its file format
2. Build a terrain editing tool into the game’s editor (this will require a bunch of work to do a good job, and will have fewer features than an external tool but they may be more appropriate for your game; also, you can make reasonable things happen to entities placed in the world when you interactively move the terrain around)
3. Paint heightmaps in an image editor like Photoshop and hotload those from the game engine (requires much less programming; doesn’t provide many features, but you have very fine control over the results; unfortunately, the editing process becomes pretty cumbersome)

These seemed to be the main approaches on offer.  However, I wasn’t enthusiastic about doing any of the traditional things; I have written a number of terrain systems for games, and what bugged me, with respect to this game, was the way in which terrain handling tends to be very different from laying out objects in the world, and how this creates an extremely disjoint work process.

In concrete terms, this means that terrain was not going to be as malleable as the other objects in the world.  I am taking a very rough-drafty approach to the game; we are building areas, then moving them across the world if we feel they will work better somewhere else, rotating them as a group, etc.  The goal is to build a very-cohesive game world in which carefully-tuned gameplay is the most important thing — where we more-or-less have a rough draft of the whole world set up before we go in and try to get serious about how everything looks.

This kind of approach just dies when you have elements that behave very differently, and are difficult to manipulate together, such as terrain and objects.  If I have two hills with a path winding between them, a bench by the path with a book upon the bench, and a signpost nearby, what happens when I want to move that scene across the map and then rotate it by 30 degrees?  With the entities, it is no problem, we just do that in the editor.  But with the terrain it just becomes a mess to move the hills at all, regardless of which of the above approaches are used.  And if you want to move and rotate the hills in sync with the entities so that the whole scene is intact… good luck!

To solve this problem, terrain needed to be built in the in-game editor, in a way that would be maximally malleable — so that I can pick up a hill and rotate and drag it.  Because of the requirement for rotation, you can’t really be manipulating samples directly; it has to be control surfaces of some kind (because if you are just blitting samples around, a few rotation operations are going to mangle whatever original shapes you made).

As for control surfaces, the first ideas that come to mind are things like spline patches.  But the problem with spline patches is that you have to be very careful managing the control points in a patch, and they ultimately aren’t that malleable — what if I want to split a patch in half without changing the shapes of areas near the edges?  What if I want to move one small, high-detail terrain area so that it overlaps somewhat with a big, low-detail area?  How is that going to work? And if I have spline patches, then any time I instantiate a new grid of patches, I have to manage a bunch of individual control points, which is labor-intensive.  If I am rough-drafting, I want to do the minimum possible work to lay out a given area.  If I just generally want to build a hill but don’t care much about its shape, I ought to be able to do that with just 2 or 3 control points, arbitrarily-placed.

Clearly what I wanted was a system that would just look at a set of unstructured control points — no topology or anything — and generate a terrain from those points in the local area.  When there is no topology or parameterization, you can merge and split groups of control points at will; you can have them overlap; you can do whatever you want.

I started putting together an ad hoc scheme for doing this, but did not take it very far before discovering that the mathematics for doing a good job at exactly this problem had already been invented decades ago.  The method is called Kriging, and it is well known in geostatistics.  If you’ve got a survey of height field data (or any other field data — the density of a mineral at certain points, for instance), and that data is missing values for some locations, Kriging will give you a reasonable guess at the values of the field at the unknown points.

For rough-draft terrain modeling, we can turn the idea around; if we give a Kriging function only a small set of known data points, it will do a quite fantastic job of filling in the values at all the other points.  The basic math involves the inverse of an n by n matrix, where n is the number of known data points. In theory that could become a huge computation, but for the number of points we may typically have in one local area of the map, the computation is easy for modern CPUs. Kriging is not hard to implement; I started with some public domain code written at a university and modified it for our needs (though this basically involved rewriting it, as it was messy and had some serious bugs!  If anyone wants our version of the Kriging code, drop me a line).

Here’s how we have it integrated into our game engine.  There’s a type of object that exists as a set of terrain control points, which can be moved around the world arbitrarily, just like anything else.  It looks like this:

The only meaningful data in this object, with respect to terrain shaping, are the positions of the control points (purple boxes).  The surface is just a visualization; it tells you what surface these points would define if they were the only control points in the universe.  (Though actually, the surface defined by those points would extend infinitely in all directions; this visualization stays within the convex hull of the control points, which helps you see what you are doing when editing).

Right now these control points haven’t actually been applied to the game’s terrain (green surface below); they are just hovering above it.  Here’s a shot from another angle:

I can press a key to add a new control point to this object:

The new control point doesn’t affect the shape of the old surface in any way, which is very convenient. I can drag the new control point at will:

Or add many more control points:

Here’s that same visualization surface, with the control points and gridding turned off, so you can see it better:

Then, when I decide I like the shape, I can press another key to apply these control points to the game’s terrain:

(Now there’s a lot of z-fighting between the control surface and the terrain because they are in pretty much the same place.)

Here is the terrain with the control surface hidden.  Note that the terrain values have been smoothly interpolated, even in areas outside the control object:

(By the way, those seams you see in the previous two pictures are lighting seams.  These don’t have anything to do with Kriging, it’s just that the terrain blocks don’t know how to look up their neighbors’ height values when generating their surface normals.  Any block-based terrain system has to solve a similar problem.  It’s easy to solve, but at a low priority on the to-do list, hence it hasn’t been done yet.  The distance between seams tells you how big the terrain blocks are, so the seams are sort of useful in this image!)

Suppose this control surface is part of a scene that I want to reorient.  I’ll rotate it about 70 degrees clockwise (now you’ll see the surface intercutting more-heavily with the existing terrain values):

And then once again, I apply Kriging to the terrain, generating the same hills in a rotated orientation:

To illustrate the way that interpolation is not confined to a single control object, and that in fact this is a control point soup, here I’ve created three control objects:

And here’s what the terrain looks like with them applied:

And here’s the final terrain along with the control objects (lots of z-fighting again):

Hopefully that’s enough screenshots to illustrate the point.  Compared to splines, Kriging has the following highly beneficial properties (aside from the huge benefits of not requiring a topology, discussed above):

• Always interpolates every control point exactly
• Adding a new control point does not change the existing surface at all.
• No funny spline-like behavior (curves appearing in strange places, huge overshoots, etc)

 

Hacking In Compact Support

When doing Kriging in a video game, you probably don’t want to compute the terrain for the entire world at once for every change, because that would be very expensive, and it also is a little bit unintuitive (since a change to a control point very far away could change some interpolated terrain nearby, even if only slightly).  However, the vanilla Kriging algorithm would require global computation and thus behave nonlocally.  For The Witness we clearly wanted our control points to affect only nearby areas, i.e. have compact support. My first attempt at this was:

• Pick a radius r0 at which any control point has maximum effect, and a larger radius r1 at which the point has no effect, with smooth interpolation in-between.
• Terrain consists of contiguous blocks 33×33 samples in size.
• For each block of terrain you want to Krige, find all control points whose distance in the XY plane from any sample in the block is less than r1 (these are all the control points that could possibly affect any sample in the block).
• Pass all these points to the Kriging routine, which computes every sample in the block.  It does this by building an inverse variogram matrix — square with dimension (n+1) — and performing one matrix-vector multiply for each terrain sample, which is pretty fast.
• For each sample, the Kriging routine gives us the set of weights we would use to blend each control point if the control points did not have compact support.  To provide compact support, just compute an additional factor that is (distance – r1) / (r0 – r1) clamped to [0, 1], multiply all the weights by these factors, and then re-normalize the weights (ensure they all add to 1).

This almost works, but not quite; we get discontinuities between terrain blocks (which should not happen if everything is correct, since two vertices at the same XY position should always get the same Z value). At first I thought this was a programming error, but after looking more closely at the situation I convinced myself that the reason is mathematical.  My assumption had been that a control point with a weight of 0 will have the same effect as a control point that never enters into the calculation, but this seems not to be true.

The step that I added, multiplying the compact-support factors against the weights provided by the matrix-vector multiply, is — by the distributive property — the same as multiplying each row of the matrix by the support factor before doing the matrix-vector multiply.  If a support factor is 0, then this makes the matrix rank-deficient, i.e. it could not have been the result of an inversion, which means that this step is not compatible with the mathematical technique being applied.  The idea of a 0 blend factor just isn’t valid.  (This also brings into question whether the idea of multiplying by the support factors is valid at all, but for now I will just observe that it produces very useful and convenient behavior.  However, there are some small stepping artifacts that I see in some locations.  I think these may be due to having too-low a density of control points in the world (some terrain samples having 0 control points inside r1) but there’s a possibility that they aren’t, and have something to do with the weighting procedure itself.  I haven’t investigated yet.)

Now, as I type this, I am not 100% convinced that it is the right reasoning, and that something more like a typical programming error might be the problem.  My math is pretty rusty these days.  However, my updated version of the routine does not have discontinuities.  

Unfortunately, the fix for discontinuities has made the Kriging a bit slower.  What I do is, instead of building just one inverse variogram for the entire terrain block, I build a new variogram for every vertex, and include in that variogram only the control points that have nonzero support factors.  Thus the dimensions of the matrix will change from sample to sample, and I am inverting a new matrix for each output sample.

Unoptimized, this version of the algorithm takes between 5 and 25 seconds to recompute the terrain for the entire world, depending on how large r1 is.  This is fine for now, especially since editing operations only hit about 1/50 of the world or less at any given time.  However, it’ll get slower as more control points are added to the world.

Fortunately, this first version of the discontinuity-fixing was intended to be slow — I programmed it in the most straightforward way possible, to maximize the likelihood that it would work.  It is easy to think of ways to make it faster by caching the matrices, but it’s unclear which of these would be fastest.  For example, if you just cache one copy of the matrix and invalidate it any time a new control point comes into or out of range, then you might easily find yourself in a situation where you oscillate between recomputing two different matrices as you work your way along the sample grid.  This is a little reminiscent of the kinds of problems that 3D hardware has, which store images in a non-linear way so that they can be cache-efficient when sampled along lines in arbitrary directions… but it’s not clear to me how far to take that analogy or whether any applicable wisdom can be derived from it.

So I intend to optimize it, or come up with a faster solution to the discontinuity problem, but it’s not clear yet what form that will take.

]]> http://the-witness.net/news/2010/05/kriging-is-cool/feed/ 27 http://the-witness.net/news/2010/05/the-island-this-week/?utm_source=rss&utm_medium=rss&utm_campaign=the-island-this-week http://the-witness.net/news/2010/05/the-island-this-week/#comments Fri, 07 May 2010 17:31:20 +0000 Jonathan Blow http://the-witness.net/news/?p=337

]]> http://the-witness.net/news/2010/05/the-island-this-week/feed/ 18 http://the-witness.net/news/2010/04/the-island-so-far/?utm_source=rss&utm_medium=rss&utm_campaign=the-island-so-far http://the-witness.net/news/2010/04/the-island-so-far/#comments Wed, 28 Apr 2010 00:25:55 +0000 Jonathan Blow http://the-witness.net/news/?p=328 I figured it would be fun to post occasional screenshots of the island so people can see it gradually come together over time.  Here’s the first one.

Keep in mind that this is all a rough draft for gameplay purposes, and none of this is supposed to look remotely like final game visuals.

Despite not looking fancy yet, there’s already more than 5 hours of gameplay.  So that’s pretty good.

(Click image for full size.)

]]> http://the-witness.net/news/2010/04/the-island-so-far/feed/ 26 http://the-witness.net/news/2010/04/graphics-tech-shadow-maps-part-2-save-25-texture-memory-and-possibly-much-more/?utm_source=rss&utm_medium=rss&utm_campaign=graphics-tech-shadow-maps-part-2-save-25-texture-memory-and-possibly-much-more http://the-witness.net/news/2010/04/graphics-tech-shadow-maps-part-2-save-25-texture-memory-and-possibly-much-more/#comments Wed, 21 Apr 2010 21:41:09 +0000 Jonathan Blow http://the-witness.net/news/?p=113 In the previous part, I talked about my reasons for wanting to use Cascading Shadow Maps (one of the biggest: image stability), then said that we had implemented Michal Valient’s version of the algorithm, and mostly liked it, but wanted to reduce the memory usage.

This time I’ll show how to reduce the memory usage by 25% on machines that support non-power-of-two textures; or, on machines that don’t, how to fit 5 shadow maps in the space previously used for 4.

Valient’s shadow system generates very stable shadow maps, but this costs a lot of texture memory.  Recall that so much memory is needed because you are inscribing your frustum slice inside a circle, and then inscribing that circle in the texture map:

When implementing cascading shadow maps, you will be using more than one frustum slice: say, four or five.  Valient’s ShaderX 6 paper suggests a way to sample from four different shadow slices quickly in the shader by packing them into an atlas, perhaps like this:

This was the way we implemented it at first, but very soon after that I was desiring a little more versatility.  If your graphics hardware will do non-power-of-two textures, then you may wish to have the number of shadow slices be an adjustable property; but if the texture maps are packed into a square like that, it’s hard to know where to put a fifth one.  So we changed the atlas layout to this:

This still works on power-of-two hardware (power-of-two textures don’t have to be square!), and on more-versatile hardware, we can now easily add new textures onto the end.

When your atlas is laid out in a line like this, and you add some runtime visualization to your game, it becomes very clear how much texture space is going unused.  Here’s a shot from The Witness‘s shadow maps:

The frustum slices are shown in green.  Obviously there is a lot of empty space between them.  (For the purposes of visualization, we have left the pixels in the “empty space” filled with shadow map data, but anything that is not covered by green will never be visible in-game, so the game will run faster if we cull all that stuff!)

Because there’s so much empty space, we could move our shadow maps closer together, and use less memory.  But we need to be careful about it, because the whole point of the algorithm is that we need enough shadow map space to handle the worst case.  Indeed, if you turn the camera a bit, you get this:

Now there is very little space between the maps, so you can’t squeeze them together.  Instead, all the space is toward the bottom of the atlas, and it is unclear how to use it effectively.

But there’s a trick we can apply.  Even though these shadow maps are all packed into one atlas, they are still indexed as separate units in the shader, which means we can transform them as separate units.  So why don’t we just rotate each individual shadow map by 90 degrees?  Here’s what happens to the image above when you do that:

Now there is plenty of horizontal space and we can pack the maps all together!  (Of course we don’t need to render our shadow maps at the default orientation and then copy-and-rotate them in a post-pass; we can just rotate the up-vector of the camera used to render the shadow maps, so that we get the rotation for free.)

When we rotate the shadow maps like this, in order to render properly, our shader needs to realize that they are rotated; but Valient’s system already provided a way to transform the shadow map coordinates based on map index, so rotating the maps fits directly into that scheme.  Furthermore, if we are using any filter offsets to index into our shadow map, we need to ensure that those offsets get rotated (by either rotating the constants before uploading them to the shader, or by adding them before doing the map-to-atlas transform, depending on how your system is implemented).

So how do we figure out how to pack the frustum slices together?  Our implementation slides them to the left as far as we can make them go.  First, we compute bounding rectangles for the frustum slices (bounding rectangles shown in yellow):

These rectangles are all oriented in the same direction, so we can just slide them to the left until they nearly touch:

We figure out the rectangle orientation by performing a brute-force uv-bounding computation on one of the frustum slices, then using that same orientation for all slices.  By “brute-force uv-bounding computation” I mean that we just try a whole bunch of different angles and see which one fits the tightest.  In the current implementation we try 100 different orientations spread over 90 degrees, so we find the best angle to within less than 1 degree of accuracy.  (You only need to try 90 degrees, because at that point every angle has been covered by at least one of the sides of your rectangle).

The initial instinct was to use Shadow Map 0 to compute the rectangle orientation, but this didn’t quite work out; because of the extreme pointiness of the first frustum slice, we sometimes end up with bounding rectangles that do not suit the other maps.  Using any of the other shadow maps solves this problem (because their near plane tends to create a coherent bounding edge).

We’re not done moving the shadow maps around.  Besides just shifting the later shadow maps leftward, as above, we save some more space by shifting the first shadow map leftward based on the leftmost vertex in its frustum slice.  (We don’t do it based on the yellow rectangle, because that rectangle will almost always be poking outside of the atlas already.  We don’t need to make sure the whole yellow rectangle is inside the atlas; just the green frustum slice!  Remember that the rectangles are just a tool to help us know how to pack.)  So then we get this:

And we can shave off even more space by shifting the rightmost map upward or downward, whichever direction will allow it to slide further to the left (in this case, it’s downward:)

Clearly we’ve saved a lot of space — about half for this viewing angle. But we have to allocate enough texture space for the worst case, and the question is, what is that worst case? Empirically, for our game’s FOV, we find that in this worst-case viewing angle we can chop off 25% of the texture map.

If you’re on non-power-of-two texture hardware, you can just cut your map down and you are happy.  If you are power-of-two-only, you can’t do that, but you could use five frustum slices in the space that previously held only four.

Dealing with Overlapping

You can pack these rectangular areas together however you want, as long as they don’t overlap — at scene rendering time, your shader doesn’t know or care how close these regions are to each other.  Actually building the shadow maps in the first place has become a bit trickier, though.  When each shadow map was in its own self-contained square region, we would just set up a viewport for that square, erase the contents, and render into it.  But now, suppose we are rendering the shadow maps from left to right: while we are rendering the second shadow map, its viewport is going to inevitably overlap the first shadow map.  We have to be careful not to write into the valid region of that first shadow map, or else we’ll mess up the scene.

In our current implementation, we use DirectX9′s API-provided clipping planes for this purpose.  (The long sides of the yellow rectangles tell us exactly where to put our clipping planes. )  Clipping planes are deprecated in future versions of the API, so we would either switch to stencil testing or else do the clipping ourselves in the shaders.

Here’s what the neighborhood of the worst case looks like:

It happens when you are looking sort-of in the direction of the light vector, but you are turned enough to the side that the view frustum has rotated to be diagonal relative to the axes of the texture map space.  (If you are pointing directly along the light vector, your view frustum extrema are likely to be parallel to the axes of the texture map space.)

Deeper Thinking We Have Not Yet Done

We’ve saved this texture memory by doing what are essentially image-space operations.  But it seems to me that one could somehow project these operations back to the 3D space that we use to compute the bounding spheres in the first place, and generate smaller spheres.  The operation of rotating our texture maps means that we are folding our output space over on itself, using only half the possible orientations.  The whole point of the bounding sphere is to permit the use of any orientation at all — which we clearly, in retrospect, do not need.  So it’s probably possible to reformulate this algorithm in terms of a bounding hemisphere of some kind (or some shape that is somewhat near a hemisphere but allows frustum corner protrusions to poke out of it).

Saving Tremendously More Memory, if you are willing to copy your maps or else render them in multiple pieces…

The reason our system works is that once the shadow maps are packed into an atlas, the shader doesn’t know or care where the boundaries of the shadow maps are.  We can address the atlas at will, because in reality it is just one continuous texture map.  This means that we can go even further, and place some of our shadow maps in the atlas in such a way that they cross the boundaries of the atlas and wrap to the other side.  It looks discontinuous, but the shader being run at scene-render time does not care, if the texture is addressed in a wrapping mode.  Here is a messy mock-up (not the output from an actual running version, because we haven’t implemented this):

Unwrapped, easy-to-see version of texture space:

Wrapped vertically:

Wrapped horizontally:

In my done-by-hand mockup, the result is 46% the size required by the default packing in Valient’s original algorithm.  That’s a lot of memory savings.

The catch is that, while it’s easy to employ this mapping without additional expense, it’s more expensive to generate.  You have to either render a non-wrapped version of each shadow map and then copy it into its multiple destination pieces, or else you have to render each map in multiple passes.  (It’s always possible to arrange your maps so that one of them is unsplit, so for 4 frustum slices, you probably need 3 additional passes.  Each pass has fewer objects, though, so maybe it’s not too bad.)

If you are using pre-filtered shadow maps, then the filtering step is just a kind of copy anyway, so you would be able to do this packing essentially for free.  We aren’t doing any pre-filtering on The Witness, but if we do, it is likely we will switch over to this packing scheme.  The memory savings are so huge, how could we not?

But I’d like to reiterate for clarity that at this time we have not implemented this wrapping-shadow-map scheme, so if there are any hidden gotchas, I don’t know.  We have only implemented the above 25% savings scheme (sliding the maps around, no wrapping).

]]> http://the-witness.net/news/2010/04/graphics-tech-shadow-maps-part-2-save-25-texture-memory-and-possibly-much-more/feed/ 27 http://the-witness.net/news/2010/04/graphics-tech-multi-lightmaps/?utm_source=rss&utm_medium=rss&utm_campaign=graphics-tech-multi-lightmaps http://the-witness.net/news/2010/04/graphics-tech-multi-lightmaps/#comments Mon, 12 Apr 2010 00:47:26 +0000 Jonathan Blow http://the-witness.net/news/?p=260 Ignacio has implemented the first version of the handling for light sources that change (doors that open or close, lights that can be switched on or off, etc).  The idea is that we just precompute different lightmaps for each of these cases and blend between them at runtime.  In the future (not the future of this game, but the general computer graphics future) when we have realtime global illumination, this would not be necessary; but for now this is much faster to compute and much easier to implement.  The drawback is that we have to think about the different cases in advance (things can’t be too dynamic) and the amount of lightmap storage space grows rapidly as the number of variables grows.

We don’t yet smoothly interpolate between the maps in the shader; we just switch the maps outright.  Interpolation is coming soon.  But in screenshots you can’t see changing conditions anyway, so it’s time for some screenshots.

First is the house with the door closed, then with the door open.  After that comes the sculpture room, with two light sources toggled in various ways (see the captions on each picture). For now you’ll have to forgive the light leaking in the corners and the shadow speckles on the walls; they are due to issues with the dynamic shadow system (which is not really related to what is being shown here) and those issues have yet to be resolved!

[Show as slideshow]

]]> http://the-witness.net/news/2010/04/graphics-tech-multi-lightmaps/feed/ 25 http://the-witness.net/news/2010/03/graphics-tech-texture-parameterization/?utm_source=rss&utm_medium=rss&utm_campaign=graphics-tech-texture-parameterization http://the-witness.net/news/2010/03/graphics-tech-texture-parameterization/#comments Tue, 30 Mar 2010 08:05:57 +0000 Ignacio Castaño http://the-witness.net/news/?p=120 As explained by Jon in his previous post, we are using precomputed global illumination in The Witness, and one of my first tasks was to work on that system.

I have developed some interesting technology to compute automatic paramaterizations, to formulate the lighting computations in a way that can be easily mapped to the GPU, and to speed them up performing them at a lower frequency by using irradiance caching. None of these things are very cutting edge, but I run into some interesting problems that are usually overlooked in the literature and came up with some creative solutions that I think would be interesting to share. However, in retrospect, I think I also made some mistakes, but I hope that being honest about them will help me avoid them in the future.

In this first post I will describe the lightmap parameterization algorithm that I have implemented and in the next few posts I’ll cover other details of our lighting system such as our approach to perform final gathering using hemicubes and our implementation of irradiance caching.

In order to use precomputed lighting we need a way of mapping these lighting computations to the surfaces. One possibility is to store them per vertex, but in many cases that does not provide enough resolution and the interpolation of the lighting makes the triangulation of the mesh fairly obvious. Many games get away with that by compensating the lack of detail in the lighting with high texture and normal map detail, but that does not go along well with the graphical style that Jon wants to achieve. So, instead of that we decided to create full parameterizations of our objects in order to store the precomputed lighting into lightmap textures.

Creating these parameterizations by hand is possible, but it is time consuming and I’d rather have our artists work on something more enjoyable. So, we decided to create the parameterizations automatically. However, the greatest mistake I’ve made so far is to not look more carefully at the available solutions to accomplish that. I had some pieces of the implementation already available, and I overestimated how much effort it would take to put them together and produce robust results. While talking about this to some friends at GDC one of them asked why I wasn’t using Microsoft’s UVAtlas tool, and to my dismay and embarrassment my only answer was that it had not occurred to me. I think that goes to prove how valuable it is to be open about our development in a small team like ours.

Now that I got that out of the way, let’s talk about the tech. We only use lightmaps to represent indirect illumination and the direct illumination of static area lights. So, our expectation was that lightmaps would be fairly smooth and that seams would not be a huge problem. For this reason we decided not to invest in a solution that guaranteed seamless parameterizations and instead used a fairly traditional approach: We first divide the input geometry into charts with disk topology, parameterize them individually, and finally pack them into a rectangular texture.

Segmentation

Dividing the mesh into charts is by far the most complex step of the process. In the past I had tried using hierarchical clustering without much success, so this time around I decided to use an iterative clustering algorithm similar to Lloyd’s algorithm, in the same vein as the algorithm used in the Multi-chart geometry images (MCGI) paper.

Like Lloyd’s clustering algorithm, the MCGI algorithm iterates between chart growing and reseeding stages. The growth of the charts is guided by a metric based on the normal deviation and the distance between the face centroids. After each chart growing iteration a new seed is selected taking the most central face of the chart, and the process is repeated until the seed location does not change anymore.

My implementation of the MCGI algorithm worked moderately well, but it didn’t always produce good results. It was possible to tweak the number and location of the seeds to obtain the desired output, but that required a certain amount of user intervention that was not desired.

Part of the problem is that the meaning of “good” in this setting is very subjective. It usually means nicely shaped charts, that fit well to the surface, produce parameterizations with little distortion, have fairly straight and convex boundaries and are more or less equaly sized.

In order to adapt the algorithm to our purposes I did two things, modified the bootstraping method so that the initial number of seeds is chosen automatically and used a combination of metrics to obtain better control over the results.

In order to select the initial number of seeds, I simply insert the seeds one by one, and grow the respective charts until a threshold is met. At that point I pick an arbitrary face that is not assigned to any chart yet and repeat the process. When no faces are left, the growing process is restarted from all seeds in parallel.

A common strategy to remove the sensitivity to the initial conditions is to use the farthest point heuristic. However, instead of doing that I found that an approach that works well is to pick the worst candidate face of the already chosen charts as the next seed.

The second important change to the algorithm was to use a different set of metrics. The MCGI paper proposes a metric that combines normal deviation with distance between face centroids. I found that using absolute distances made the algorithm very dependent on the tessellation of the mesh and that measuring the distances from the centroids favored triangles with low aspect ratios.

Instead of using distance between faces I considered only the normal deviation and combined it with other metrics similar to the ones proposed in D-Charts: Quasi-Developable Mesh Segmentation and that take roundness and straightness into account.

These metrics also had their own problems and I had to tweak them in various ways. The most significant issue is that they constrained the chart growth in undesired ways. For example, to encourage chart convexity you would associate a high factor to the corresponding metric, but that would result in a large number of almost convex charts, as soon as a convex configuration was found no additional faces could be added to the chart. So, instead of using these metrics to constrain the growth I only use them to favor convex charts or straight boundaries; that allowed the charts to grow freely, but still retain the most convex and most straight shape possible.

A trick that helped in many cases was to incorporate some input from the artist, but without imposing any additional burden to them. A simple way of doing that is taking into account the existing parameterization of the mesh. Our objects already have a parameterization that is used for texturing and the artists tend to place texture seams at locations that are not easily visible. Snapping the boundaries of our charts to the existing seams did not only reduce the overall number of seams, but also caused the charts to evolve in a way that matches better what the artist would have done manually.

I considered texture and normal seams independently. Since our parameterization is used for lighting I follow normal seams more strictly and proportionally to their angle, not doing so would cause the corners to appear less defined or more blurry. That also helps the process to converge much more quickly in architectural meshes that have many hard edges.

Finally, I don’t impose any chart topology constrains. I specially wanted charts go grow around holes in the mesh as long as the metric was below the desired threshold. I was tempted to use a better proxy fitting metric instead of the normal deviation metric, but that would have generated cylindrical or conical charts that would have to be cut, and I did not want to deal with that. Closing holes on the other side is fairly easy, it’s just matter of detecting the boundaries and closing all, but the longest one.

Parameterization

For the chart parameterization algorithm we simply use Least Squares Conformal Maps since I had an implementation already available. My implementation is based on the description of the algorithm in the notes of the SIGGRAPH 2007 Mesh Parameterization course, which I found much more accessible than the one in the original paper. Particularly useful is the chapter about numerical optimization, although if I had to write a sparse matrix solver again I would probably use OpenNL, since it has now been released under a liberal BSD license.

LSCM is not the best parameterization algorithm out there. It worked pretty well in most cases, although in some rare situations it resulted in small self-overlaps. That happened in two cases:

• Zero area triangles in the input sometimes flipped in the parameterization.
• Charts with complex boundaries that self-intersect when parameterized.

The first case turned out that was caused by T-junctions in the input mesh that were detected as holes and filled with a degenerate triangle. In practice the small overlaps in the parameterization were not really an issue so instead of fixing the input we simply plan to report these problems to the artist in the exporter and have him fix the them if it turns out it’s causing artifacts.

In the second case, increasing the weight of the convexity metric used during the clustering process allowed us to prevent these errors. So, I didn’t really do anything to handle this problem, but so far it has not been an issue.

Even after that we had some problematic cases remaining. I noticed that the implementation in Blender was not having this same issues and was handling these cases more gracefully. So, I asked Brecht Van Lommel, who suggested the use of alternate matrix weights based on the ABF formulation of LSCM, and that seemed to eliminate the remaining issues.

While LSCM is very easy to implement, it’s definitely not the best parameterization method out there. If I were to implement something better I would probably look at LinABF or Fitting to Guidance Gradient Fields. However, since parameterization of the individual charts is currently the fastest step it may make sense to also look at more expensive deformation analysis methods.

Packing

Our packing algorithm is fairly simple. Like many other implementations we first sort the charts by parametric area, rotate them to align the charts with their best fit rectangle and greedily introduce them in the atlas one by one. Ideally I should test all 4 possible orientations, but currently I simply pick one of the orientations randomly.

Instead of using a tetris-like scheme to find the best location for each chart, I use a more brute force approach in which all the possible locations are considered. I achieve that using a conservative rasterizer that tags all the texels touched by a chart, plus an extra texel for padding. I progressively increase the texture extents as new charts are inserted. If the a location within the current extents is found, the chart is automatically accepted, otherwise a metric is evaluated to determine the location that least increases the extents of the atlas. Since I use a greedy algorithm, minimizing area resulted in atlases with very high aspect ratios. So, instead I minimize a metric that combines extent and perimeter and results in mostly square atlases.

Here are some examples of the results:

Once we use lightmap compression it will probably make sense to align the charts to DXT block boundaries in order to prevent bleeding artifacts.

Future Improvements

This is one of the areas where I think there’s a lot of room for improvement, but at the same time, there’s not much !/$ in spending more time improving the results. Even though it has not been tested in production, I think that our current solution works fairly well. It would be nice to have larger charts with less distortion or a lower number of seams, and I would certainly have fun working on that. However, that’s not going to make the game run significantly faster or look dramatically better. We plan to release the source code of these algorithms at some point so my hope is that the implementation will progressively get better as other people adopt it and improve it to fit their needs.

That said, if there’s one area where I think it’s worth investing, that is in the use of signal specialized parameterizations. The research papers on the subject (Signal-specialized parametrization and Signal-specialized parameterization for piecewise linear reconstruction) are probably overkill in our setting, and I’m not sure they would work very well with our low resolution meshes.

Half-Life 2 achieved that that by using artist specified sampling rates for each surface, but I think it’s worth doing that automatically using an approach like the one proposed by Bungie, which seems fairly simple and provides most of the gains the more complex approaches would give us.

My expectation is that the current implementation won’t suffer significant changes, other than small tweaks and slight adjustments to deal with problematic cases that may show up in the future. If someone has any experience with these kind of algorithms, I would love to hear about it. In particular I’d like to know if there’s anything you would have done differently, or if there’s any optimization or improvement that I may have overlooked and you think is worth considering.

]]> http://the-witness.net/news/2010/03/graphics-tech-texture-parameterization/feed/ 9 http://the-witness.net/news/2010/03/graphics-tech-lighting-comparison/?utm_source=rss&utm_medium=rss&utm_campaign=graphics-tech-lighting-comparison http://the-witness.net/news/2010/03/graphics-tech-lighting-comparison/#comments Mon, 29 Mar 2010 06:08:20 +0000 Jonathan Blow http://the-witness.net/news/?p=196 It seems like time to post some images from the in-progress precoputed lighting tech that Ignacio is working on.  See here for a description of the basics behind what we are doing for lighting.  The lightmaps are now 16-bit and these shots use a simple exponential tone mapper.

First, I’ve taken shots from 3 positions inside a house, with four different lighting settings.  For all four settings, we are computing only the first lighting bounce.  The parameters are:

1. 32×32 cube maps, adaptive sampling, quality = 0.5.  Time to compute: 197 seconds.
2. 32×32 cube maps, non-adaptive sampling.  Time to compute: 648 seconds.
3. 64×64 cube maps, non-adaptive sampling.  Time to compute: 738 seconds.
4. 128×128 cube maps, non-adaptive sampling.  Time to compute: 1570 seconds.

(Click for full size).

Front Door:

32×32 adaptive

32×32

64×64

128×128

Corner:

32×32 adaptive

32×32

64×64

128×128

Side Wall:

32×32 adaptive

32×32

64×64

128×128

The hardware used to compute these timings is an Intel Core i7-860 with an ATI Radeon 5870 (most of the process is GPU-bound).

The non-adaptive sampling renders one cube map for each shadow map texel, whereas the adaptive sampling uses heuristics to place samples and then smoothes the map between them.  At higher quality settings, adaptive sampling provides more-accurate results, but there always seem to be artifacts, so for the time being we are using it as a fast preview mode.  If at some point the quality becomes shippable, then we would switch to adaptive sampling for everything.

We also have a sub-texel sampling mode that improves quality, but I didn’t play with that during this run.

This second set of images shows progressive bounces of the lighting with 64×64 cube maps, non-adaptive sampling.  (The first one is the same 64×64 image as above).  The times are: 1 bounce, 738 seconds; 2 bounces, 1438 seconds; 3 bounces, 2481 seconds.  The precompute time is basically linear in the number of bounces, as one might expect (the algorithm is basically just doing the same thing n times.)

1 bounce

2 bounces

3 bounces

]]> http://the-witness.net/news/2010/03/graphics-tech-lighting-comparison/feed/ 10 http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/?utm_source=rss&utm_medium=rss&utm_campaign=graphics-tech-shadow-maps-part-1 http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/#comments Wed, 03 Mar 2010 18:22:04 +0000 Jonathan Blow http://the-witness.net/news/?p=77 The Witness contains a mixture of indoor and outdoor scenes, but much of the game takes place outdoors with a very long view distance (you can see the entire island at once if you have a good vantage point).  So I wanted to implement a shadow system that would work robustly, provide high visual quality, and allow the player to see everything at once.  I have some experience with shadow systems of this type, but the last one I designed was for computers and graphics cards circa 2004, so I was interested to see how much more would be possible today.

We’ve implemented such a modern shadow map system for The Witness.  In the process, we’ve made some improvements to shadow mapping algorithms beyond anything we’ve seen published, so we are going to detail the improvements here.  Also, our shadow map system is still being improved, so I’ll talk about what we have yet to try and why we think it’s a good idea.

Before we get to those details, though, I’d like to establish some context so that the motivation for these design decisions is clearly explained.

The Literature

I have a somewhat cynical attitude toward graphics research literature: most of it describes techniques that don’t generally work, but the authors of the papers do the best they can to “sell” the technique to you anyway (using cherry-picked examples, glossing over or completely ignoring failure cases that would be obvious to anyone who understands the algorithm, etc).  As the reader, eventually you come to understand all the problems, but not after investing a lot of your time and energy (possibly months) implementing and understanding an algorithm that behaves so poorly that you never would have bothered if you had known the truth from the outset.

I’ve had this experience many times, with many different techniques.  Shadow maps, though, have been one of the big ones.  There are many published shadow map techniques that simply don’t work well enough to be taken seriously.  And often when I’ve heard someone say “so-and-so shadow technique is good,” it usually turns out they haven’t tried it themselves, so it’s just hearsay, or else that person has low quality standards.

So I’d like to put forth the statement that I have high quality standards and will only endorse things that have been found to robustly function; I will be open and honest about the degree to which things don’t work, and what the specific problems are.  In an ideal world this would not be necessary to say, but the situation in the literature today makes it otherwise.

Quality Goals; Previous System

Many shadow map schemes have been developed that try to maximize the effective resolution of shadows in the scene by performing transformations that are heavily view-dependent.  An extreme example of this is Perspective Shadow Maps.  I have learned through experience not to use these techniques.  They cause shadows to swim and flicker in annoying ways, and many of the algorithms break down severely as the player approaches certain viewing angles.

For the 2004 system, I took as a core design goal that shadows should appear rock-solid on nonmoving objects, regardless of any viewpoint motion.  The clearest way to achieve this was to center the shadow map on the viewpoint at all times, never letting the shadow map scale or rotate.  Because a single shadow map cannot cover the world at high resolution within memory and fill constraints, I used a scheme where 4 or more shadow maps of increasing worldspace size were centered on the viewpoint like square doughnuts.  In order to prevent crawling or shimmering, one just ensures that shadow map worldspace positions are snapped to integer multiples of their texel size.  (A family of related techniques, but which don’t necessarily center the maps on the viewpoint, soon came into more-common use and took on the moniker Cascaded Shadow Maps.)

On fixed-function pipeline hardware, this scheme was never quite satisfying (I had to use clip planes to render the scene in many slices, and there were small 1-pixel artifacts due to the resulting imprecision; rendering all the slices was a bit slow).  Modern hardware is able to do this kind of thing much  better.  Also, this scheme wastes a large amount of shadow map memory, because with all the shadow maps centered on the viewpoint, most of the map texels are going to be out of view at any given time.

Despite the drawbacks, the visual stability of this technique, and its ability to reach across the entire game world, were extremely appealing to me.  Having seen how nicely shadow mapping could behave in practice, this visual stability became a very-high-priority goal in my mind for any future shadow systems.

So, going into this new system, the goals were (listed in approximate order of importance):

1. High performance
2. Complete stability under camera motion
3. Long view distance
4. Visuals can be controlled to suit the style of the game
5. Efficient use of texture memory

The New System

I started working on the new system by looking at the 2004 system and trying to make it more memory-efficient.  Most likely this would involve moving the shadow maps around in world space, but it wasn’t initially clear to me how to do this without introducing problems.  Ignacio pointed me at Michal Valient’s article “Stable Rendering of Cascaded Shadow Maps” in ShaderX 6, which was exactly what I wanted.  Valient computes bounding spheres around the slices of the view frustum that tell him how much he can move the shadow maps in world space without introducing gaps.

To illustrate, here are a couple of figures reproduced from his article.  I don’t want to unduly step on anyone’s copyright, so if you are interested in cutting-edge shadow map techniques, buy ShaderX 6!  (Click for full size.)

So basically, you take the a frustum slice in worldspace, ensure that it is completely enclosed in a sphere, and then ensure that the sphere is completely enclosed in a square cylinder; the square is your shadow map.

You can render multiple frustum slices for multiple shadow maps, so long as the bounding spheres overlap enough to cover the whole frustum when put together (see 4.1.2 c and d).

The reason they are spheres is: because the shadow map is never allowed to change size (we voluntarily imposed that constraint in order to get solid shadows!), then we need to find a shape that conservatively encloses any possible orientation that a frustum slice could occupy as the camera rotates in space.  That’s a sphere.  Then we make sure that our shadow map covers that entire sphere, and we have then guaranteed that every point in the view frustum is covered by a valid shadow map  texel.

On top of this, Valient suggests the very helpful optimization of packing all your shadow maps into one atlas texture, so that when it’s time to render the scene, you can draw all your shadowed objects in one pass without having to sample multiple textures; you just figure out which frustum slice each pixel lands in, then use that information to determine the offset into the atlas, add that offset to your texture coordinates, and sample the texture.  This works great.  Valient suggests a 2×2 arrangement of textures, as this is convenient on a wide variety of hardware, for example, GPUs that only support power-of-two textures.  So if a single shadow map would be 1024×1024, then you can create a 2048×2048 texture map that contains 4 shadow maps packed in a 2×2 array:

So my first shot at a new system was basically a reimplementation of everything Valient describes.  It didn’t take too long, and when it was done, I was very happy with it — it was clearly much better than the 2004 system.

But…

Memory Usage

This technique is still a memory hog.  The image stability constraints, which result in us wrapping the frustum slice in a sphere and then the sphere in a box, add margins of unused and barely-used texture space at each step.  Looking at Figure 4.1.2, you can see that a frustum slice only occupies about 50% or 60% of the area of the square that represents your shadow texture.  This implies that half the square is wasted.  However, the actual situation is worse than this, because the diagram is misleading.

The problem is that the view frustum represented in the diagram is much narrower than the view frustum used in an actual game, and if you re-draw the diagram in realistic proportions, it looks very different.  For an accurate 2D diagram, you are finding a circle that encloses the widest part of your view frustum, which is the 2D trapezoid you get by cutting your frustum in half diagonally (Valient discusses this in his paper as well).  Suppose your game is rendering at a 16:9 aspect ratio, and your field of view is 90 degrees horizontally (this is what we use for The Witness currently.)  The vertical field of view is then going to be about 59 degrees, and the diagonal field of view will be about 98 degrees (click on image below for explanation).

The frustum slice in Valient’s diagram is only about 30 degrees, a huge difference!  So if we have a 98 degree frustum slice, and inscribe that in a circle, and inscribe that circle in a square, what does that look like?  Something like this:

Recall that the square is your shadow map texture and the innermost trapezoid is your frustum slice (the texels of your texture that may potentially be used).  It covers only a small  area of that square — and there’s no way to make it bigger!  The reason is that for wide frusta like this, the diagonal at the far plane is so long that it dominates the bounding sphere computation, and so the center of the circle has to land on that diagonal so that its diameter can be just barely large enough to enclose it.  (In The Witness, our frustum slices are not proportional, and the way we divided them up, this isn’t exactly true for the first 2 shadow maps — but it is very close to true).  The fact that the circle is centered on the far plane means that automatically half the map is wasted at this orientation — but the wide angle ensures that much of the other half is wasted too!

You can rotate the view frustum to other orientations and make it look like you are covering more of the shadow map… for example, if you orient the frustum so that the view vector is going straight down into the page, then the frustum’s projection onto the paper will be a rectangle, and it will appear to cover much more of the square.  But then you have to keep in mind that some of this coverage is worth a lot more than other coverage in terms of impact on the scene: think about how much space is covered by the texels toward the middle of the frustum projection, versus how much is covered toward the edges (almost none!)

If I add two more frustum slices, so that the total is 3 slices as in 4.1.2, it looks like this:

That innermost shadow map is just a tiny smudge (now you know why Valient chose a narrow field of view — for clarity of his figures!)  But notice what else is going on: each shadow map is fully contained within the next-larger shadow map, as with the concentric-square-doughnut system from 2004!  Valient’s scheme is better since it gives you more view distance (the squares are not concentric) — but it can’t give you nearly as much view distance as it would like, due to the wide angle of the frustum.

For Next Time

So, whereas on one level I was very happy with this shadow system’s performance and visual quality, it still seemed somewhat wasteful in terms of memory usage.   Next time I’ll talk about how we addressed that problem.  Subsequent postings after that will talk about issues like softening the shadow border, blending between slices, and various other implementation tricks.

To be continued.

]]> http://the-witness.net/news/2010/03/graphics-tech-shadow-maps-part-1/feed/ 22 http://the-witness.net/news/2010/03/announcing-indie-fund/?utm_source=rss&utm_medium=rss&utm_campaign=announcing-indie-fund http://the-witness.net/news/2010/03/announcing-indie-fund/#comments Tue, 02 Mar 2010 20:34:18 +0000 Jonathan Blow http://the-witness.net/news/?p=80

Today we are announcing a fund that provides very good terms for talented indie developers working on high-quality games.  Our terms are much friendlier than what publishers offer.  Our goal is to help indies become stronger while retaining their independence.

The past couple of years have been good for independent game developers.  Through download services like Xbox Live Arcade, the PlayStation Network, WiiWare, and Steam, independent developers have found a very large audience.  Some of us have been lucky enough to develop hit indie games that were very financially successful.

Braid was one of those games.  The success of Braid has allowed me to undertake more-ambitious projects like The Witness.  At the same time, I felt that I wanted to do something with the profits that would help other indies with their own games.  More recently, while talking to publishers about The Witness, I felt that the business climate around publishing and funding these smaller games had not caught up with reality: it’s a model where the standard terms are tuned for budgets in the tens of millions of dollars.  Because publishers want to stick to this model even for low-budget games, it was very hard for indie developers to get a fair deal.

It turns out that other successful indies felt the same way, so we have pooled our resources and launched this fund.  We will be announcing further details soon.  At the GDC, Ron Carmel will be giving a talk about the problems that exist in the current publishing model for indie-budget games.

We’ve had the timing of this announcement planned for a while, but it seems like some kind of strange synchronicity that we’re revealing our existence just as all this trouble is happening at Infinity Ward.  When you’re a mainstream developer, and you’ve made one of the most successful and profitable games of all time, and then just a few months later your publisher and parent company is willing to so bald-facedly mutilate your company, well, what conclusions can be drawn from that?  If publishers of that size are so megalomaniacal as to be incapable of seeing the importance of a developer’s talent — instead believing that the game’s success is somehow due primarily to their brilliant marketing strategy or their CEO’s charming personality — then how will this ever change?

If Infinity Ward can’t be treated with respect, then who can?

Independent developers can.  That’s one answer, at least.  Indie Fund is here to help make that independent existence a reality for as many talented developers as we can.

Here’s a link to the main Indie Fund site, with an email you can use to contact us.

]]> http://the-witness.net/news/2010/03/announcing-indie-fund/feed/ 6 http://the-witness.net/news/2010/03/twitter-and-facebook-pages-are-now-live/?utm_source=rss&utm_medium=rss&utm_campaign=twitter-and-facebook-pages-are-now-live http://the-witness.net/news/2010/03/twitter-and-facebook-pages-are-now-live/#comments Tue, 02 Mar 2010 07:37:17 +0000 Jonathan Blow http://the-witness.net/news/?p=71 You can now be notified of new blog postings via RSS, Twitter or Facebook.  See the icons in the upper-right corner!

At the moment of writing this post, The Witness has no Twitter followers and no Facebook fans, so you could be the first!

]]> http://the-witness.net/news/2010/03/twitter-and-facebook-pages-are-now-live/feed/ 2 http://the-witness.net/news/2010/03/graphics-tech-precomputed-lighting/?utm_source=rss&utm_medium=rss&utm_campaign=graphics-tech-precomputed-lighting http://the-witness.net/news/2010/03/graphics-tech-precomputed-lighting/#comments Mon, 01 Mar 2010 00:40:45 +0000 Jonathan Blow http://the-witness.net/news/?p=50 For the past 15 years or so, graphics technology in games has been driven by shooters.  Most shooters generate visual interest in their scenes by having lots of dynamic lights, with extreme use of bump mapping, and moving shadows, and particle effects.

For The Witness, I wanted to develop a graphical style that values simplicity.   It would be suited to mellower environments, with both indoor and outdoor settings.  Some kind of global illumination solution seemed like the right idea here.  (Global illumination is when you simulate light bouncing around in a scene; the resulting look is usually much richer and subtler than the direct lighting that games usually do.)

Clearly, full real-time global illumination would be the most versatile solution.  I investigated some licensable packages that provide this, such as Geomerics’ SDK.  However, these solutions will invariably use a lot of texture space, and consume a bunch of processing time, and they seem like overkill for this game (The Witness does not have many moving light sources, relative to other types of games).

So, some form of precomputed global illumination seemed like the right thing.  3D modeling packages have plugins to compute global illumination, but they are very difficult to interface with, and they could not even come close to handling a full game scene.   The only thing that knows where all the objects in the world are at once is the game itself (and the in-game editor), so it seemed appropriate to develop our own in-game system for global illumination.  It could have been radiosity, could have been something like Monte Carlo Path Tracing, but Charles Bloom suggested the very simple solution of just rendering the scene for every pixel in a lightmap, and that seemed like a good idea.  With an approach like that, you don’t have to do a bunch of monkeying around to make radiosity or ray tracing match the completely different algorithm that is used to render your realtime game scene; light transport is computed by that same algorithm, so it will automatically match unless you mess things up.

For several months, Ignacio Castaño has been working on this system.  He is writing up a highly technical explanation of how the system works, which I think he’ll be done with in a few days, but in the meantime, here is a lighter-weight overview.

At preprocess time, we just walk a camera over every surface, pointing the camera away from the surface and rendering the game scene into a hemicube in video memory.  The render target textures of a hemicube are packed together into an atlas, and when the atlas fills up, we download it into system memory, integrate the values in each hemicube to find an average light value, and then store that value in the lightmap.  Most of the shots below were generated using 128×128 cube maps and multiple samples per texel.

Because we are just using our regular rendering code, this precomputation process is hardware-accelerated by default.

Note: In the images below, all geometry and textures are placeholders. As with Braid, we are using rough-draft versions of all these things while the core gameplay is built.  Bloggers, these are not to be considered screenshots of the game.  They aren’t supposed to look good yet — they are just supposed to show the precomputed lighting.

First, some shots of the house.  The precomputed lighting is responsible for most of the light in the room.  The places where you see sunlight hitting surfaces directly are dynamic; everything else is a static precompute.  (Without the precomputed lighting, the entire room would be black except for the patches of sunlight).

Note the ambient occlusion, soft shadows, and other nice effects that all come naturally from this system. (In the third image, the picture on the wall hasn’t been lightmapped yet, which is why it sticks out so brightly.)

Here’s a simple bunker interior consisting mostly of a single material and color.

Lastly, the tower:

There are lots of caveats on these images.  The game renders in HDR, but we haven’t balanced our lighting constants or done tone mapping yet, so I have gamma-corrected the output images by various amounts.  For now, we are computing only the first lighting bounce.  We’ll probably do at least 2 or 3 bounces for the final game, and when we have that running I’ll post some comparison images.  I believe our model of sky illumination is still overly simple and wrong (though I’d need to check with Ignacio on that, maybe he has fixed it up).  I haven’t played around with the results of the current system in outdoor areas (e.g. lots of small branches and leaves), though if we find that we have too many aliasing problems, we can always make the lightmaps higher-resolution in those areas.

In areas that have doors or windows that open and close, we are planning on layering a supplemental lightmap that gets scaled by the openness of that light aperture, then added to the base lightmap.  This will provide a pretty good approximation for the way the indirect light in the room changes as the aperture opens and closes.  Direct light will be completely correct since it uses the fully dynamic shadow system.  I’ll do a post about that shadow system sometime in the future.

]]> http://the-witness.net/news/2010/03/graphics-tech-precomputed-lighting/feed/ 30 http://the-witness.net/news/2010/02/concept-art/?utm_source=rss&utm_medium=rss&utm_campaign=concept-art http://the-witness.net/news/2010/02/concept-art/#comments Sun, 28 Feb 2010 05:58:18 +0000 Jonathan Blow http://the-witness.net/news/?p=9 In order to begin discussing the game, it seemed more appropriate to show some concept art, because the game is very visual.

Click for full-sized versions.  The gallery seems to hang some versions of Google Chrome.  Sorry about that.  It works fine in Firefox and Internet Explorer.

[Show as slideshow]

These concepts (and many more) were all done by Eric Urquhart, who has been working on the game for some months now.

]]> http://the-witness.net/news/2010/02/concept-art/feed/ 39 http://the-witness.net/news/2010/02/welcome-to-the-development-blog-for-the-witness/?utm_source=rss&utm_medium=rss&utm_campaign=welcome-to-the-development-blog-for-the-witness http://the-witness.net/news/2010/02/welcome-to-the-development-blog-for-the-witness/#comments Sun, 28 Feb 2010 04:06:16 +0000 Jonathan Blow http://the-witness.net/news/?p=4 The Witness is a game in development, designed and produced by Jonathan Blow, designer of the critically-acclaimed game Braid.  There’s a small team working on the game, and over time this blog will introduce you to them and provide a bit more insight into the development process.

Many people have been asking for more details about the game, so I thought it might be nice to kick things off with some concept art… in the next post.

Welcome, and thanks for your time and attention!

-Jonathan.

]]> http://the-witness.net/news/2010/02/welcome-to-the-development-blog-for-the-witness/feed/ 13