mirrored local space normalmaps

CrazyButcher

just a little proof that it works
3dsmax exporter (as well as converter from other formats) of next luxinia release will support generating the necessary per-vertex attributes.
I also intend to make a little tutorial about it + showing that it can work with deforming meshes, too. though it will take a while until then.

the mirroring works on any arbitrary axis, ie you dont have to use major axis. The only restriction I am currently aware of is, that making two-sided geometry is illegal (ie I need a bit of volume between sides to detect the mirror plane).

Mirrored parts must lie outside 0-1 and must be shifted by multiples of 1. So that the fractional UV part ( what is after the comma) is still the same as the reference side. This restriction is only because I wanted it to work from any input format, if the technique was directly wired to the dcc exporter where more information about triangle connection and so on exists, another method for marking "outsiders" could be used, but I think this works fine. Afterwards outsiders are actually moved back to 0-1.

runtime performance costs are slightly higher than straight local maps, but less than tangent.

As for which format compresses better / has less artefacts on compression (tangent vs local), I dont have proper source art and so on at hands (too little experience on the whole baking stuff). In theory however there are more texture formats for tangent compression I think.

I'll bump this thread again once the release and more info is out

edit:
http://developer.nvidia.com/object/real-time-normal-map-dxt-compression.html
probably the best article about compressing normalmaps, also hints towards better possibilities with tangents.

Eric Chadwick

Awesome!

The way I understand the compression issues...

1. With a tangentspace map you can disregard the blue channel (calculating it in the pixelshader), so you only have to store red and green. If you want lossless 8bit per channel, then it's only two channels vs. the three for objectspace.

2. DXT compression of a tangentspace map without the blue channel yields less artifacts because it takes advantage of the fact that the green and alpha channels have higher precision. Also the compressors do better if red and blue are the same uniform value (per 4x4 chunk, a separate oddity that could be exploited!) so basically for the best results you want to store only two channels of data, thus tangentspace wins over objectspace if using DXT.

CrazyButcher

yeah I know the theory about those, just not how the "day to day" useage is like who is actually using object/local space normalmaps and do they run into compression issues...
is the only reason they dont use it the mirroring thing or other stuff ?

in those charts the "average" signal noise ratio is better for tangents (but also has a few where it's worse than object/local space).

Joao Sapiro

holy jesus man thanks ! if implemented to game engines this would end the era of having to add edges for the tangent space map to work!

EarthQuake

The main problems we've had with it are mirroring, and complications when trying to combine painted normal detail back on top, but we have sort of a solution for this. We've really came to the conclusion that NM compression basicly sucks no matter what we do, and we just selectively turn it off for cases that are really bad and half that texture so we dont have memory isssues. Generally if we have a texture that compression is raping, droping it to 512 from 1024 and turning off compression will give us better results.

Eric Chadwick

To add painted detail, are you basically just bumping the lowpoly with a combo of tangentspace map (converted from grayscale paint) and objectspace map (baked from mesh), and baking that combo into objectspace again?

EarthQuake

We have a tool that lets you load up your reference mesh, and object space map. Then load up your tangent normals, and just some sliders for things like tile and amount. We need to load up a mesh to know how to correctly orientate the tangent normals or else things will come out upside down or reverse etc. It mostly works, but it tends to "bend" the resulting normals, so you gotta split the mesh up into some smoothing groupds before you run it, and then i usually will just composite this "combo" texture over my orig map in PS.

Murdoc

Alright, a little lost here since most of this (what Earth was telling me in his weapon thread an dnow this) I didn't think was all that possible.

What benchmark are you guys defining for performance, PC or consol?

Earth: The tool you're refering to is a method to layer or combine tangent space detail maps on object space normal maps? or do you just use it in your 3d package to amke sure the nothing is upsidedown and then just bake it across to be unique rather then tiling in the in game material?

In what game context are you working in, FPS? ( not that this is too significant, I'm just curious)

CrazyButcher

I didnt benchmark. but I ordered the "likely" performance by thinking of what resources (math + pervertex and perpixel vectors) each method needs for bare minimum to work.

Of course when shading is more complex and more resources are spent, the hit of changing between each might be very tiny.

Afaik the game EQ works on had support for mirroring for a while. I now have found time to update tools for luxinia as well (which isnt used commercially for games stuff atm), I've mentioned the idea for mirroring them last year I think in this forum as well. The math involved is really simple, which surprises me that its used so rarely, but I would blame better compression support for tangent stuff.

edit:
should add that "compression" does not only have the benefit of less memory, but (which is much more important) means faster texture sampling. which does speak for tangentspace and 3dc texture compression

Eric Chadwick

Thanks EQ.

CrazyButcher

got some time to work on viewport shaders for max again, and also worked on script to generate the needed per-vertex data. Yeah I know last time I teased a tutorial and didn't deliver, but this time will provide the shader and tools and integrated into 3dsmax.

SHEPEIRO

coolio

Eric Chadwick

CrazyButcher, did you ever get around to releasing this shader? We're working on something similar here, and I was curious how similar our approaches were.

Also I was curious for any info you had about using a localspace map with deforming meshes.

EarthQuake

CB has some tools (scripts + integration into 3ps shader) working for this, i think he's doing some work now to get that + the custom tangent space stuff packaged up.

Eric Chadwick

Thanks EQ.

CrazyButcher

hehe yeah should never make promises you cannot hold The "raw code" is fairly simple and straight forward. A working version has been around in 3ps for quite some time, but its not really of interest here, as clients use classic methods. Hence I never saw the need to push it out again, given the work associated with making a nice package with better workflow, but through the custom TS stuff I can add it more easily to max now.

the algorithm is really straightforward:
- you compare uv chunks (inside and outside 0-1 box) and compute the mirror normal for corresponding uv verts. that you do in preparation step and store per-vertex.
- on run-time at vertex level, you pass that mirror normal (n) (0,0,0 for stuff that isn't mirrored) and good old reflection math will then either reflect the texture normal (v) or not. No special care must be taken as passing 0,0,0 will simply not change the input vector. (reflect = v - (2 v dot n) n)

assuming world space lighting, deformation would be analog to tangent space (in TS you deform the TS vectors before passing to pixel stage). But now you pass the deformation matrix (without translation) directly and run them on the flipped normal.

Eric Chadwick

Thanks CB!

Will pass this along to our rendering programmer. We're doing normalmapping on the Wii, pretty amazing it can actually do it since it doesn't use pixel shaders at all.

We're leaning towards a mix of dot3 normal mapping and embm mapping, since embm can support both object and tangent maps. Of course tangent maps compress better, can tile, are easier to edit, etc. But... embm only uses an environment map for lighting, so you either accept static lighting, or you have to render the env map dynamically. We'll see...

BTW, I found the Nintendo patent for EMBM, an interesting read.

CrazyButcher

ah wii, I think its fixed function pipeline will not allow that much arithmetic ops on the pixel stage, so you guys probably will flip the light direction in vertex-stage and then pass to pixel.

Eric Chadwick

Yeah pretty funky hardware, so our rendering programmer says. Actually she said it's pretty powerful, can do a lot, once you wrap your head around the different system it uses. Still, we developed our last game for PC and Wii at the same time, which was frustrating at times, pixel shaders are just so much easier to work with in comparison.