Normal map... What am I doing wrong?

rockbox

hey folks,

i baked a normal map for this model and am getting some visual artifacts, as you can see here:



It's doing something weird on the bottom across the line of symmetry too.



I followed this thread: http://www.polycount.com/forum/showthread.php?t=107196

and this video: https://www.youtube.com/watch?v=kGszEIT4Kww

First, I had the lowpoly on one smooth group and baked and the normal map was crap. I seperated all the UVs that had different smoothing group for hard edges in the lowpoly.

after I finished I offset the mirrored UVs 1 unit, and when I exploded for baking, after attaching all low poly objects, I reset the xForm on it before exporting it to xNormal and baked using a cage. and yet, it still looks like crap. better crap...but still crap.

edit: oh, and in xNormal normal map settings the y is -y. I'm using 3ds max with xoliulshader and tried flipping the y in max, but it doesn't look any better. in fact, I don't even know if it should be flipped or not they look nearly the same.

gsokol

So you are using the Xoliul shader?

What version of Max? If its 2013 or 2014...then that is the problem. Autodesk made some changes to how tangents are handled in the viewport and basically made most dx shaders, including the xoliul shader, completely useless.

sprunghunt

3dsmax sometimes does some weird things across the line of symmetry even though the normalmap is fine. I find that if you put the same model into udk, unity, or some other engine it will render fine.

rockbox

Yes, I am using max 2014 with the direct3D render settings because i didnt like the quality on some images that the nitrous setting was giving me.

I did put it in marmoset and it looks loads better, but still has some black in the corners of the windows

EarthQuake

In Toolbag 2, make sure you set the tangent space (in the mesh properties) to Max, assuming you've baked the normal map in max.

rockbox

negative, I baked in xNormal. the latest update.

EarthQuake

Theres an Xnormal tangent space setting too, though the synced results with Max bakes look a little better.

rockbox

I understand that the bakes will look different between programs, but what you're saying is that a map baked in max will look better in max correct? but, will the map be a better quality map in one program or the other or is the only advantage to using xNormal is that it can give quicker bakes?

EarthQuake

rockbox wrote: »

I understand that the bakes will look different between programs, but what you're saying is that a map baked in max will look better in max correct? but, will the map be a better quality map in one program or the other or is the only advantage to using xNormal is that it can give quicker bakes?

The whole situation is a bit of a mess.

First off, most 3d apps/normal map bakers use slightly different math (bi-normals and tangents), so if you try to take a normal map baked in maya and render it in max (or vice versa) you'll get errors. Idealy you want your target app/engine to use the same math (generally called "synced tangents").

In the newest versions of Max, autodesk broke some stuff, which prevents correct display of normal maps in the viewport (offline rendering should work ok), previously you could use Xoulio's shader or 3point shader of accurate normals.

In Maya, if you use HQ viewport, the normal map accuracy is actually quite good (again, for maps baked inside of maya). Though in the recent versions, you want to turn off face weighted normals, as again, Autodesk adding new "features" and breaking stuff.

Toolbag 2, unlike other apps, supports multiple tangent spaces, so if you bake in Max, Maya, or Xnormal your normal maps will all look very good (Max and Maya slightly better, there is one issue we still have with the XN bakes).

You can also bake a tangent space map in any baker, and use handplane to convert it to a variety of tangent spaces.

rockbox

Ok that makes sense. so game engines will also use different math,similar to toolbag 2, in order to accommodate these different programs? I remember cryengine used to have their own tangent basis calculator for xNormal, but that is not longer valid in the new xNormal 3.18.6.

EarthQuake

Game engines will often use different tangent basis', totally proprietary ones, which is where handplane is very useful, its the only way to convert your map to certain engines.

rockbox

I see. well this has been very helpful EarthQuake, I'm gonna do a little more research on this Handplane and see how it fits into a cryengine workflow.

rockbox

EarthQuake, you said there is still one issue we have with xNormal bakes.. which issue are you referring to?

Brandon.LaFrance

Earthquake, somewhat off topic, but I'm curious. What application outputs a normal map that uses Marmoset's default tangent basis? Or rather, is there a way for xNormal to output a normal map in Marmoset's default tangent basis?

Not a huge deal, but when I'm blocking out my materials, I like to separate my model in to different pieces depending on material. That way I can just play with sliders to quickly get acceptable RGB values for my varying maps before ever touching Photoshop, which can save me a ton of time. I could save a bit more time if I didn't have to go in and change the tangent basis for each mesh beforehand.

Kimon

Brandon, nope and probably no.
That's the problem with tangents, there's no standard what so ever.
All applications uses their own and some, like 3ds max and UDK, are a real pain in the arse sometimes

If you get problems I think handplane is your best bet, if you don't have your object space normals aviable you can always convert your tangent to object with xnormal. I havn't actually tried that last part, but I know the tool is there!

rockbox

Ok I redid alot of things, adjusted the hi poly and I exploded the low and hi but then I added an unwrap modifier to the exploded low to offset the symmetrical half 1 unit and now my bakes are all messed up, why is this?

EDIT: Nevermind, I fixed it. When I reset the xForm it changed the position and it didnt line up with my cage and hi poly anymore.

ZacD

Post pictures of what results you are getting. One sentence doesn't give us much to go on.