Hard Edges When Rendering With Normal Map

kmacneil91

Hey guys!

I'm having issues rendering my normal maps in both Max and Maya. I seem to get the hard edges seen in the photo below, regardless of reversing normals, softening edges, setting normals to face, etc. I've tried rendering with VRay and Arnold. I did the retopo in Togogun and baked out the normal map there, too. The weird thing is that when I bring the mesh into Substance Painter and use the normal map there, it looks completely normal. Any ideas?

Thanks!

2bytes

In Maya, make sure the bump node is in Tangent space

huffer

Theres a different tangent space between viewport and software renderer...it might never be good. But does the normal map look ok in the viewports of max and maya?

kmacneil91

Yes, the node is set to tangent space, thanks. The normal map looks great in the Maya viewport, using the quality renderer. In Max it doesn't seem to show up in the viewport.

2bytes

If it looks correct in the Maya viewport,  then you will need to use the correct Vray node for rendering normal maps, and then use the correct Tangent Space (left or right) which simply flips the Y channel (green).  Also, make sure the file node is using the RAW colorspace.  Also,  when rendering geometry that uses normal maps, do not subdivide the mesh at rendertime...this will create artifacts in the render because the normal map will not match the geometry normals after subdivision.

kmacneil91

2bytes said:

If it looks correct in the Maya viewport,  then you will need to use the correct Vray node for rendering normal maps, and then use the correct Tangent Space (left or right) which simply flips the Y channel (green).  Also, make sure the file node is using the RAW colorspace.  Also,  when rendering geometry that uses normal maps, do not subdivide the mesh at rendertime...this will create artifacts in the render because the normal map will not match the geometry normals after subdivision.

Thanks so much for the information! A few questions:

-I only have VRay for 3Ds Max currently. How would I check the node there? And is there something I need to do with the nodes for Arnold in Maya? (Any screenshots you may have would would be super helpful)
-For the Y channel, I can just invert this in Photoshop, right? 
-RAW colorspace - how do I check that in either Max or Maya?
-I haven't subdivided the geo, but just use soften edges.... not sure if that makes a difference

Much appreciated. 

kmacneil91

Never mind guys! A coworker helped me figure it out. Just in case anyone else has this issue:

In Max, I had to use a gamma override of 1.0 on the normal material. I also had to fix the normals on the geo itself using this script:

http://mixescript.blogspot.com/2013/03/fix-meshes.html

In Maya, I exported the corrected mesh and used smooth preview or made sure to use catclark subdivision smoothing with enough iterations.