converting local to tangentspace normalmaps
polycounter lvl 20
when improving the normalmap support for luxinia, I stumbled upong the problem of normalmap creators using different tangentspaces. Which means if your game doesnt use the same you are doomed... well slight errors will occur.
This is mostly an issue for max8 as it uses non.orthogonal tangent space, which is more accurate, but also less gametypical (well for the new stuff probably not anymore either).
Someone had similar problems with max8 and asked on gdalgos, he later found a solution. Use localspace (should be the same for all apps, at worst some channel flipping) and convert it to tangentspace.
I spend the last days to integrate such a tool into a pipeline commandtool, which also converts 3d formats to luxinia's format.
The outcome is pretty useful
It also overcomes the problem of using mirroring, which would be a problem in local space, but as they are converted, the issue should be gone.
The tool uses MESA OpenGL software rendering combined with vertex/pixel shaders and basically renders an ortho view of the mesh in uv space. The shaders take care of the conversion. They are not hardcoded and other shaders could be used. Also use of MESA means its all CPU, so independent of capability of GPU.
the shading of the hipoly and lowpoly might not fully match in the picture, due to me having not much experience with normal maps and setting it up properly, plus the subdivision was higher on rendertime. (and the lights are different). Nevertheless for the hardcase of having a simple hard edged box as lowpoly, it did work well enough.
the next version of the engine itself will actually making normalmapping useful, as last public release is a bit buggy.
The tool is very oldschool commandline, I will integrate a GUI version in the engine, which calls the tool and will handle the command args.
I am so happy it works, cost me some nerves, that I wanted to share
This is mostly an issue for max8 as it uses non.orthogonal tangent space, which is more accurate, but also less gametypical (well for the new stuff probably not anymore either).
Someone had similar problems with max8 and asked on gdalgos, he later found a solution. Use localspace (should be the same for all apps, at worst some channel flipping) and convert it to tangentspace.
I spend the last days to integrate such a tool into a pipeline commandtool, which also converts 3d formats to luxinia's format.
The outcome is pretty useful
It also overcomes the problem of using mirroring, which would be a problem in local space, but as they are converted, the issue should be gone.
The tool uses MESA OpenGL software rendering combined with vertex/pixel shaders and basically renders an ortho view of the mesh in uv space. The shaders take care of the conversion. They are not hardcoded and other shaders could be used. Also use of MESA means its all CPU, so independent of capability of GPU.
the shading of the hipoly and lowpoly might not fully match in the picture, due to me having not much experience with normal maps and setting it up properly, plus the subdivision was higher on rendertime. (and the lights are different). Nevertheless for the hardcase of having a simple hard edged box as lowpoly, it did work well enough.
the next version of the engine itself will actually making normalmapping useful, as last public release is a bit buggy.
The tool is very oldschool commandline, I will integrate a GUI version in the engine, which calls the tool and will handle the command args.
I am so happy it works, cost me some nerves, that I wanted to share
Replies
also i didnt want to write tons of exporters for normalmap creators, so let them use localspace and convert that...
its not my idea, I just saw that someone else had similar problems on gdalgos-list and he solved it after getting some advice,too, so I went on and did it his way.
Also, would there be any way to change the way Max renders out normals to match those required by a different engine (such as Doom3?)
and yes it's the binormal. orthogonal systems often compute it out of normal and tagent, to save bandwidth. non-ortho however is slightly more accurate afaik, but requires normal,tangent and binormal to be sent, which likely isnt that much of an issue on latest hardware and bandwidth anymore...
I kinda doubt a simple texture format-conversion is going to remove all your shading seams. We've really had to tackle it at the vertex level. And it still isn't perfect, requires some artist input, designing a UV set that tells our algorithm where to put the seams because we can't eliminate all seams, just the major ones (like the face midline).
Cool idea though, sounds interesting.
The convertor does require the mesh and all per-vertex attribs, it also needs all mirrored UVs to be of "same side". As the local space normal map of mirrored stuff can of course contain only data for one side, and it is crucial for the conversion that the proper one is chosen. The conversion itself is done on per-pixel level using the same interpolations that would be used on rendering... (thats why MESA is used to resemble 3d hardware shading)
Though as the ortho space I use is not as accurate as the non-ortho of max or others, I guess issues can arise, at least another coder warned me that ortho space has it issues too. however that wouldnt change the workflow of the method, as any normals/binormals... could be sent.
I tested with a noised and deformed box,
http://crazybutcher.luxinia.de/wip/local2tangent/
the first screenshots are converted from local , the max ones use the max tangentspace normalmaps.
to be fair the error must happen as different tangentspaces are used. So for me this conversion is needed to stay somewhat independent from max's way of making tangents.
however one can also see seams slightly in my converted version, that is due to ortho not as good as non-ortho... but I think with some better and "closer" hipoly mesh (the noised hipoly box was far away at times..) I could hide them a bit better. likely they will never go away... it's a big improvement for me anyway.
if you are after fixing those small glitches while sticking to ortho too, then thats out of my league and as you mentioned would required more work in tangentspace creation. (but to me it looks as if those issues could be fixed by using non-ortho space)
btw another code told me to recreate max8's tangentspace have a look at the VNormal.cpp (samples\materials\NormalBump) which is what they use, but means dependency on max8
Looks like your solution is coming along well. Lighting is an improvement, but still those pesky lines.
by the way I just thought of another issue, when geometry lines are not same in pixel-length, wouldnt that also mean interpolation runs non-perfect ?
But anyhow, even if you export the tangents that Max creates, you still have trouble wrapping a 2D flow across a 3D surface. You're going to have to have some seams somewhere on the surface where the tangents don't face the same direction.
Poles of a sphere are the most obvious example (and most difficult to solve apparently)... what happens to the tangents as you run over the top of the sphere and start heading down to the equator again? Usually all the tangents at the pole face "Up" at the pole, so you get a seam there. Anyhow, not something I worry about, almost never UV that way.
Hmmm. Think of an arm meeting a torso. How do you solve all the tangents there? If the arm and torso are each treated as cylinders, then the arm's "up" needs to be reconciled with the torso's "up" along their shared vertices (the seam). The tangents there end up quite different.
And for fixing the seams this just wont work the way I thought, so my early joy was just a bubble that exploded hehe
The only time max renders them right as well is the scanline renderer. but so far I couldnt find out why. they must be sending some data slightly different than the realtime version..
If you compare Max-generated tangent-space maps with ones generated from identical meshes in the Doom3 engine, you'll see subtle differences (I recall Eric Chadwick mentioning something about different bi-tangents on the lowpoly mesh or something?) ... is there any way that some sort of "standard" could be set for how normals are derived and used between meshes? Kinda annoying when you have different viewers and engines, and each one needs a *slightly* different tangent-space map. It'd be better if they were drastically different, but the only difference I can see is slightly more exaggerated gouraud compensation on Doom3 normals as opposed to Max normals.
Also, would there be any way to change the way Max renders out normals to match those required by a different engine (such as Doom3?)
[/ QUOTE ]
Doom3 uses some sort of hybrid tangent thing that isnt the same as max. Chai has a half-written doc about it here: http://www.svartberg.com/tutorials/article_d3normalmaps/d3normalmaps.html
gdAlgortihms has a related thread right now, might be worth a read.
http://sourceforge.net/mailarchive/forum.php?thread_id=30627501&forum_id=6188
Eric I just got the gdalgos batched email-digest today
but basically I hope the stuff I did so far is suffcient for "real world" models and not the extreme asteroid case which I artificially created...