Modern GPUs filter seamlessly across cube map faces. This feature is enabled automatically when using Direct3D 10 and 11 and in OpenGL when using the ARB_seamless_cube_map extension. However, it's not exposed through Direct3D 9 and it's just not available in any of the current generation consoles.
There are several solutions for this problem. Texture borders solve it elegantly, but are not available on all hardware, and only exposed through the OpenGL API (and proprietary APIs in some consoles).
When textures are static a common solution is to pre-process them in an attempt to eliminate the edge seams. In a short siggraph sketch, John Isidoro proposed averaging cube map edge texels across edges and obscuring the effect of the averaging by adjusting the intensity of the nearby texels using various methods. These methods are implemented in AMD's CubeMapGen, whose source code is now publicly available online. While this seems like a good idea, a few minutes experimenting with CubeMapGen make it obvious that it does not always work very well!
Embedded Texture Borders
A very simple solution that even works for dynamic cube maps is to slightly increase the FOV of the perspective projection so that the edges of adjacent faces match up exactly. Ysaneya shows that in order to achieve that, the FOV needs to be tweaked as follows:
fov = 2.0 * atan(n / (n - 0.5))
where n is the resolution of the cube map.
What this is essentially doing is to scale down the face images by one texel and padding them with a border of texels that is shared between adjacent faces. Since the texels at the face edges are now identical the seams are gone.
In practice this is much trickier than it sounds. While the fragments at the adjacent face borders should sample the scene in the same direction, rasterization rules do not guarantee that in both cases the rasterized fragments will match.
However, if we take this idea to the realm of offline cube map generation, we can easily guarantee exact results. Cube maps are often used to store directional functions. Each texel has an associated uv coordinate within the cube map face, from which we derive a direction vector that is then used to sample our directional function. Examples of such functions include expensive BRDFs that we would like to precompute, or an environment map sampled using angular extent filtering.
Usually these uv coordinates are computed so that the resulting direction vectors point to the texel centers. For an integer texel coordinate x in the [0,n-1] range we map it to a floating point coordinate u in the [-1, 1] range as follows:
map_1(x) = (x + 0.5) * 2 / n - 1
We then obtain the corresponding direction vector as follows:
dir = normalize(faceVector + faceU * map_1(x) + faceV * map_1(y)
When doing that, the texels at the borders do not map to -1 and 1 exactly, but to:
map(0) = -1 + 1/n map(n-1) = 1 - 1/n
In our case we want the edges of each face to match up exactly to they result in the same direction vectors. That can be achieved with a function like this:
map_2(x) = 2 * x / (n - 1) - 1
If we use this map to sample our directional function, the resulting cube map is seamless, but the face images are scaled down uniformly. In the first case the slope of the map is:
map_1'(x) = 2 / n
but in the second case it is slightly different:
map_2'(x) = 2 / (n - 1)
This technique works very well at high resolutions. When n is sufficiently high, the change in slope between map_1 and map_2 becomes minimal. However, at low resolutions the stretching on the interior of the face can become noticeable.
A better solution is to stretch the image only in the proximity of the edges. That can be achieved warping the uv face coordinates with a cubic polynomial of this form:
warp3(x) = ax^3 + x
We can compose this function with our original mapping. The result around the origin is close to a linear identity, but we can adjust a to stretch the function closer to the face edges. In our case we want the values at 1-1/n to produce 1 instead, so we can easily determine the value of a by solving:
warp3(1-1/n) = ax^3 + x = 1
which gives us:
a = n^2 / (n-1)^3
I implemented the linear stretch and cubic warping methods in NVTT and they often produce better results than the methods available in AMD's CubeMapGen. However, I was not entirely satisfied. While this removed the zero-order discontinuity, it introduced a first-order discontinuity that in some cases was even more noticeable than the artifacts it was intended to remove.
You can hover the cursor over the following image to show how the warp edge fixup method eliminates the discontinuities, but sometimes still results in visible artifacts:
Any edge fixup method is going to force the slope of the color gradient across the edge to be zero, because it needs to duplicate the border texels. The eye seems to be very sensible to this form of discontinuity and it's questionable whether this is better than the original artifact. Maybe other warp functions would make the discontinuity less obvious, or maybe it could be smoothed like Isidoro's method do. At the time I implemented this I thought the remaining artifacts did not deserve more attention and moved on to other tasks.
Modifed Texture Lookup
However, a few days ago Sebastien Lagarde integrated these methods in AMD's CubeMapGen. See this post for more results and comparisons against other methods. That got me thinking again about this and then I realized that the only thing that needs to be done to avoid the seams is to modify the texture coordinates at runtime the same way we modify them during the offline cube map evaluation. At first I thought that would be impractical, because it would require projecting the texture coordinates onto the cube map faces, but turns out that the resulting math is very simple. In the case of the uniform stretch that I first suggested, the transform required at runtime is just a conditional per-component multiplication:
float3 fix_cube_lookup(float3 v) {
float M = max(max(abs(v.x), abs(v.y)), abs(v.z));
float scale = (cube_size - 1) / cube_size;
if (abs(v.x) != M) v.x *= scale;
if (abs(v.y) != M) v.y *= scale;
if (abs(v.z) != M) v.z *= scale;
return v;
}
One problem is that we need to know the size of the cube map face in advance, but every mipmap has a different size and we may not know what mipmap is going to be sampled in advance. So, this method only works when explicit LOD is used.
Another issue is that with trilinear filtering enabled, the hardware samples from two contiguous mipmap levels. Ideally we would have to use a different scale factor for each mipmap level. That could be achieved sampling them separately and combining the result manually, but in practice, using the same scale for both levels seems to produce fairly good results. We can easily find a scale factor that works well for fractional LODs as a function of the LOD value and the size of the top level mipmap:
float scale = 1 - exp2(lod) / cube_size; if (abs(v.x) != M) v.x *= scale; if (abs(v.y) != M) v.y *= scale; if (abs(v.z) != M) v.z *= scale;
If you are using cube maps to store prefiltered environment maps, chances are you are computing the cube map LOD from the specular power using log2(specular_power). If that's the case, the two transcendental instructions cancel out and the scale becomes a linear function of the specular power.
The images below show the results using the warp filtering method (these were chosen to highlight the artifacts of the warp method). Hover the cursor over the images to visualize the results of the new approach:
I'd like to thank Sebastien Lagarde for his valuable feedback while testing these ideas and for providing the nice images accompanying this article.
Note: This article is also published at AltDevBlogADay.
So… This makes the game look better? I try googleing and wikipediaing almost all the things you talk about and from what I can gather: This images will be used on stuff that reflects stuff like water and chrome and so then you need them to look good and remove little artifacts created while rendering (cuz consoles) so you made parts of computers talk to each other in a different (in consoles) ways until you got it looking better and now they look good…
Is that it?
Basically, yes. It is a workaround for older hardware that improves the physical accuracy of reflections. Since much of the puzzles in the witness involve careful observation of phenomena that occur in the real world, physical accuracy is very important. That is why features such as dynamic shadows were implemented so early in development.
I may be wrong, but this is what I gather anyway.
Thank you. I do see reflections on the circles. They look like square rooms, maybe its for metal things or shiny surfaces that are in rooms or something and puzzle might be about looking at things through bounces to see a different thing. Like if you look at the reflection of something it makes a shape or what ever.
So that no artifacts interfier in the puzzles in the console version he did this… I guess…
Is there a book(s) you could recommend that would get one up to speed on the basic graphic concepts discussed here? I wish I could appreciate what was being accomplished.
Justin had a good question; how much do you use cube maps in The Witness? Is this method valuable enough to also be used when baking shadow maps? BTW, since you mentioned it, how are you baking in lighting? Is it just regular shadow maps, or will you be using BRDFs in some manner?
Using reflections and other such real-life stuff in puzzles sounds awesome. That’s what I love about games: they let you do things that you can’t do normally, whether it’s due to it being truly impossible (like in Braid) or due to the lack of resources making it virtually impossible (like in The Witness).
I mean, from what I’ve seen in this blog, all of this seems “possible” in real life. It’s just that a computer allows you to create the island yourself, put everything where you want it, even though all the mechanics simulate the natural world.
It would be interesting if there were puzzles involving cubemap reflections, but I am not certain that will be the case. Certainly the example cubemaps are too blurry to be of much use for puzzles. The general light mapping is also calculated on GPU through cubemaps, I believe. But that would make it a bit odd to worry about how they would be calculated on older hardware, since it is an offline operation to bake the light maps.
I suppose it would be impossible to know, given the secrecy of Jonathan about the gameplay of his projects. But I would love to find out why Ignacio found it necessary to make the cubemaps more accurate on older hardware.
Not necessarily on consoles, there are still many people running Windows XP, which only supports Direct3D 9. The Xbox 360 actually supports some features beyond what is possible in Direct3D 9, but is not as advanced as Direct3D 10 ( Windows Vista) or 11 (Windows 7). The Playstation 3 is a difficult analogue to a PC as it runs a different system architecture entirely, but it supports a few features not possible under Direct3D 9, if I recall properly.
Okay I hate captcha. This was supposed to be in response to Justin.
Very interesting article.
How is performance affected by using this method?
Teh vidya induztry: http://www.youtube.com/watch?v=aIP73xIAPLU&feature=g-all&context=G2a85eedFAAAAAAAABAA
I want to talk about video games, but everyone here is scared to comment for some reason? Why are people here so apologetic and hardly converse with others? Does somebody, anybody want to talk about video games? Here or we could meet at another site!
There are very strange communities to disccus games, I hate almost all of them.
Reddit, they are just in it to sound cool with or their “leleleleele” and there are the dumbest buch in the net.
Neogaf, Group of wanna-be 1337 hackerz that only care about mainstream AAA blockbusters.
Quarter to Three, Intellectual show-offs and the “2 dep 4 u” hipsters that hate games that try to be personal. In every single thread they end-up sucking each other’s dick until Tom Chick pops up and says how he hates something that is objectively: Injerently good and fundamentally worthwhile… Yeah, that guy Chick gets all of my hate!
v, This is my favorite place. totally open, heart-felt and straight forward conversations are had here with a mixture of rage, laugh, cries, hate and porn thrown in. People speak their mind and are not controlled.
Steam, They can be helpful
Any other place (RPS, Granades,Hardcore 101, Giant Bomb, The Scapist, IGN,GameFaQS, bEST gAMERS us ) is utter shit. same thing if you want Video Game Journalism™
Jonathan Blow Interviewd recently at GDC by Gamespot
http://www.youtube.com/watch?feature=player_embedded&v=I1Fg76c4Zfg
Becdause we can’t trust Jonathan to post his own interviews and share what he thinks, in his own site.
Aww shiet! Here we go again…
http://n4g.com/news/957635/jonathan-blow-explains-why-he-doesnt-play-japanese-videogames-anymore/com
2012 IGF has a new parthnership with Gamespot… Which I think is awesome because now people who are not developers can see how everything goes down… so support them so that they can keep showing the public by letting them know that we care about seeeing more stuff like this.
http://www.youtube.com/watch?v=E2kGdQUKqS4&feature=channel
48:45 Put them in the game…
Hello Jonathan! I’m one of the SMU grad students who talked to you last night after the movie screening. You suggested getting in touch with you through your blog’s comments, so here I am! The Witness is looking great, and I’m excited to see how it develops and evolves over time now that I’m following the blog. Thanks for everything!
Well… Hecker’s rant is up
http://www.youtube.com/watch?v=MMpAoLsl1QY&feature=g-all-blg&context=G28e173cFAAAAAAAAFAA
How about yours Jonathan? checking the GDC site you where supposed to give one. Hopefully is not lost!
Official recordings of the 2012 gdc sessions (albeit only some for free) will get posted (“Coming Soon”) at gdcvault.com
Hecker’s video there only exists because Hecker was cool enough to get someone to record his talk — it’s not something most people will have done.
Tom Chick does it again!
http://www.quartertothree.com/fp/2012/03/19/the-there-less-journey/
Jonathan, and you told us this motherfucker was alright. You made me follow him, but fucking hell!
He… I’m torn apart everytime he writes about a game that is trying to be deep and meaningful or that tries to be different. idk? Maybe he is trying to do something good; something to make indies learn or maybe he’s just a try-hard agent of chaos… I still like him and will keep following him… and …yeah!? Journey was indeed a bad game… He’s totally fucking right!
http://kotaku.com/5893336/jonathan-blows-the-witness-is-an-exercise-in-symphonic-game-design
cool preview of The Witness.
A note from the preview:
“And while The Witness does not appear to aspire to anything as outwardly grand as a cry of rebellion against an oppressive government, it remains remarkable how thoroughly it explores its mechanical themes.”
Does this mean The Witness is meaningless? You talk about story and the story being something you care a bout and not entertainment (anymore as it started as entertainment) but what does that mean. Braid had a story with multiple themes/subjects and they were very important and things people don’t know but if they know it could benefit the audience’s life greatly!
Braid didn’t go deep into the narrative’s meaning but the reasong the game is the best is because it has meaning or as you called it dynamical meaning in gameplay but also in story. So there is areason and purpose to do this.
The Witness will have a story. But not an entertainment story. It will be something we can use in the real world, in life and then there is the meaning and purpose of minute to minute gameplay and what you actually do. and no taking control away from the player like thatgamecompany’s Journey did with its little pointless cutscenes, right?
The Witness gameplay and story dynamical meaning and purpose will be there and will go hand and hand and add layers of complex intricate web that will help us become better and be smarter and sharper as people and we will be able to see to really see that world, right? there must be a reason why I go there, to that island. Unlike Tetris or Mario and Myst which have no purpose but more like Shostakovich’s fifth symphony; The Witness will “aspire to anything as outwardly grand as a cry of rebellion against an oppressive government”
This will be; this will exist. it must! – it will transform me, and everyone.
Totally out of subject, but you must play Journey, guys. It’s a shame that its only available for PS3, but if you are interested in arty games Journey is a wonderful experience.
Hi, I’d just like to express my gratitude for this post! I was fighting with the very problem in my still young XNA game and stumbled across your explanation and the shader solution last night. Works like a charm!
Even though the actual solution is just a quote from someone else, I think it’s great that such things are picked up by other blogs, repeated and presented in different contexts. It makes such knowledge much easier to find on search engines and often offers a different and better explanations than the original sources. Keep it up!