Normal Map Shading

jocose

I apologize if this has been answered elsewhere in the forums. I read through the pinned normal map thread but I didn't see anything that explicitly addressed my issue.

I have a model that I have been trying to normal map for awhile now but I keep on seeing what appears to be [FONT=verdana, arial][FONT=verdana, arial]per-vertex Gouraud shading[/FONT][/FONT][FONT=verdana, arial][FONT=verdana, arial] in my model.

I thought that once the model was using per-pixel shading you would no longer be able to see any hint of the underlying geometry in the shading.

So is this the case, and I am doing something wrong or am I just confused?

Thanks




[/FONT][/FONT]

Ryan Clark

Offhand, I can think of a few things things that might cause this sort of problem.

Possibility #1: It could be that your rendering program and your map-baking program used a slightly different method to calculate tangent space, and ended up with slightly different results.

To test this possibility, apply a flat blue (127,127,255) normal map to your object. Do you still see the problem?

If the problem persists with a flat blue normal map, it must be something wrong in your rendering code. See below.


Possibility #2:

If you rule out possibility #1, then it could be a problem with your pixel shader. Without knowing what your shader looks like, I can only offer a generic advice.

Make sure that you're normalize all your unit vectors inside your pixel shader. It's not good enough to just normalize them in your vertex shader, because interpolation between vertices will denormalize them, which might cause something like this.

jocose

Sorry I didn't mention that those renders are just straight out of xNormal. I didn't create a shader or anything I just passed xNormal an OBJ and a normal map and let it take care of the rest.

I tested the model with a comletely flat blue texture and I can still very easily see some of the faceting of the triangulated mesh when I move the light around.... Is this normal, or is xNormal rending things weird?

jogshy

It's hard to figure whats happening with so little information. Can you post some screenshots showing your cage, normal map, complete baking options, lowpoly/highpoly list.... and... how did you model that mesh? how was exported?

TSM

From what I understand, using Tangent space normal maps will still use the low poly's real time shading + your normal maps, however, if you use object space, then you use the high poly's shading only.

I'm open for correction.

jocose

Okay, here is an image of the high and low mesh.

Also, I loaded ONLY the model up in Unreal 3. With the default material on it I can see the exact same shading problem. So that suggest to me that it has NOTHING to do with my map, and you can indeed see some of the faceting of the model with tangent space normal maps just like TSM said.

If someone could confirm this I would appreciate it. I am just confused as to whether or not I should build my model caring about the per-vertex shading or not. I assumed since everything was per-pixel it no longer mattered and I could cut down on the number of polygons by ignoring shading problems.

Eric Chadwick

Tangent-space normal map relies on the vertex normals, but object-space does not.

If you want to use the former, you will need to adjust the topology of the low-res model. Moving some verts around can help, subdividing/chamfering can also help.

jocose

Thanks for the reply. So would it be safe to say that tangent space normal maps combine per-vertex shading and the normal map shading?

Also, I finally did get the exact result I wanted with TANGENT SPACE normal maps inside xNormal. However I am completely unable to re-produce the effect inside unreal3 with the exact same normal map and the exact same mesh.

If xNormal can take a tangent space normal map and produce the effect that I am seeing then why cant Unreal, is there some fundemental differnce, and is the faceting that I am seeing "normal" for this type of normal map?

Thanks for all your help so far.

Kawe

What was the smoothing options when you baked? What are they when you preview with normals on? They need to be the same. Also you may need to increase smoothing to remove some problems.. and gaining a few new ones. Erh.. and I'm talking about the lowpoly mesh.

This is just guess cause it is usually what gives me problems.

:poly114b: <-- this looks like the normal map smiley. beaten but still happy.

odium

Just a suggestion, try flipping the green channel. In xNormal try X+Y-Z+, then it SHOULD look better ingame.

MoP

Yes, I think Odium might be on to something, it looks like you're rendering with green being Y+, and it looks like UE3 uses green as Y- (although I might be wrong, never having used it).
Either select the Green channel in Photoshop and Invert it (Image -> Adjustments -> Invert), or change the Swizzle coords in Xnormal as Odium said.

jocose

hey guys thanks a ton,

Flipping the green channel in pshop fixed the shading in unreal. Now I am much much closer to what I am seeing in xNormal. However I still cant figure out why I am getting ZERO faceting in xnormal but I am stilling getting some in Unreal.

Once again is this "normal"? I was inclined to think that but after seeing what xNormal can do with the exact same mesh and texture I am still a little confused.

Eric Chadwick

How are you getting it from Xnormal into Unreal, what mesh format? Unreal needs to use the same bitangents as Xnormal, or the shading won't be the best. Tangents and bitangents are stored in the vertices, but are not always written/read by exporters/importers, nor are they always extracted correctly by shaders. Using the Xnormal bitangents will insure the shading looks perfect.

jogshy

However I still cant figure out why I am getting ZERO faceting in xnormal but I am stilling getting some in Unreal.

That's probably because Unreal computes the tangent basis a bit different than xNormal. I'm not sure which is their method(it's not public)... but I bet it's the one from DirectX or the NVIDIA mesh mender... try to change the default tangent basis calculator in the xNormal plugin-manager by the D3DX one or the NVMeshMender and see it helps.

For 100% perfect match you'll need a programmer though(like did 8monkeylabs with their Stooge importer).... or use object space normals instead of tangent space...

jocose

As far as the bitangents, I don't believe I am using them. I start with a base model and export that to unreal then I use that unreal file to generate the normal map in xNormal as far as I know it does not save the bitangent data.

Does anyone know if unreal even uses bitangent data for sure, and how you would go about ensuring its preserved. Would you need to then generate your maps inside your 3D package and then export them using ActorX?

Also how important are bitangents, because in Xnormal I can load up models I have exported fresh out of maya that have never had a normal map rendered on them and load them up in Xnormal with an already rendered map and they look fine.

I am really interested in understanding the topic but the results that I am seeing are telling me that you dont need them? Can somone please clarify?

CrazyButcher

the tangentspace is made of three per-vertex vectors (think axis)
normal, tangent, bitangent

if tangentsapce of "baker" and "runtime shader" dont match, you will have errors...

creating the tangentsapce is typically done by engine (and/or its tools). Xnormal supports adding custom tangentsapce generators to make sure engine and baker match.

however when you dont know for sure how unreal creates them then you are kind doomed to trial & error of different "generators".

Vailias

Also remember that unreal does respect smoothing groups / hard edges in its default import options. Not for all formats if I remember properly, but definitely for its PSK format. The other option there if you are exporting to PSK is to turn off "Bake Smoothing Groups" in the options for the ActorX export plugin.

That should help the faceting, but you are also likely to get some along UV seams as the engine sees that as a vertex split/hard edge, and I'm not certain that is accounted for in the normal map rendering of U3.