Baking in xNormals - Weird results

TheIceHeLater

When I bake normals from a highpoly to a lowpoly model in 4k resolution I always get these weird edges. Any ideas on how to get a more clean result? I have tried .tga and .tiff using the "ray distance calculator".

I was thinking it was the normals maps attempt to make the edges softer, but you can see them when applied to the model.

Scruples

Welp, I've never seen this one before, what are you using the make the highpoly? Try exporting in a different format, .Obj if you aren't already, .Fbx if you're already trying .Obj...

TheIceHeLater

Scruples said:

Welp, I've never seen this one before, what are you using the make the highpoly? Try exporting in a different format, .Obj if you aren't already, .Fbx if you're already trying .Obj...

Thank you for the advice! I make all my highpolys using zbrush. Use .obj's for import and export. I'll try fbx for the lowpoly (can't export FBX from zbrush), but I don't think that's the issue since I only export an object with smoothing data on it. FBX just incudes other stuff as well.

TheIceHeLater

As I expected, FBX didn't work

huffer

Is your highpoly highly decimated from Zbrush?

TheIceHeLater

huffer said:

Is your highpoly highly decimated from Zbrush?

Not sure what you mean by that. You mean if I have enough detail in the highpoly in zbrush?

huffer

Yup, if you're too aggresive with the decimation, and the normals are then averaged in xnormal you might get some artifacts; I usually don't even decimate and baked succesfully 10+ points meshes.

TheIceHeLater

my "lowpoly" is 134k tris and my polycount in zbrush say 4.2 mil. Should work, right? 

huffer

So you are baking directly with these two meshes? What are you previewing the baked maps in?

TheIceHeLater

Yeah, these are the two meshes. I'm previewing them in Maya 2016. It actually looks ok, but it's still bugging me. 

huffer

Then you can try another thing - triangulate the mesh in Maya before exporting to xNormal, then rebake, and reapply the map back in Maya on the triangulated mesh. The artifacts look like compensation for triangulation, and there might be a difference between how Maya does it and how xNormal does it.

TheIceHeLater

Tried triangulating the lowpoly before with no results unfortunately. 

Scruples

Now that I see the highpoly, the normal map may actually be correct if the squares are not visible in the viewport. If they still bother you, you can try changing the vertex normal calculation method.
http://download.autodesk.com/global/docs/maya2014/en_us/index.html?url=files/GUID-232E99F8-96B4-4870-8BA0-4887C1C8F0F2.htm,topicNumber=d30e189713