Normal Map UV Border Seams Even With A Cage and One Smoothing Group

Trixx

Hey Guys, I'm kinda pulling my hair out here, so I was hoping you guys could help. I have these nasty seams around every UV border on my asset, despite using a custom cage (just the actual low poly asset with pushed out faces), and one smoothing group. (Using Unreal's suggested method of a synced workflow between xNormal and UE4)

I'm using 3D coat to unwrap it, but I've imported it into Blender to confirm its using all one smoothing group and then exported from there. (I've used FBX and OBJ to export, neither seems to work).

I'm also using xNormal to bake the map, and I've set it to invert the green channel, as I'm using this in UE4, and its a tangent spaced map.

I should also note, the seam also exists in the cavity map that I've generated from the normal map, and then used as a temporary roughness map in my material in Unreal.

I've attached the pictures of the issues in UE4, my UE4 import settings, xNormal settings, and my Blender export settings. I should also note that the seams are quite apparent in both Marmoset and 3DCoat set to UE4 normal map settings. I'm at a total loss. I've read every normal map guide I could find, including the wonderful ones linked here on Polycount, but to no avail. I need help!

Edit: Here's my normal map too, in case that's useful.


.

Farfarer

Are you exporting as triangles? The triangulation has to remain identical between the baker and the game engine.

Also, for UE4, you need to enable cross product bitangent in xNormal's MikkTSpace options. Hit the plug icon in XNormal, go to the Tangent Basis tab, select MikkTSpace and then hit the Configure button. Tick the box in the options and try rebaking.

huffer

In UE4, is your imported map set to TC_Normalmap compression? UE4 sets it correctly if the file name ends with _n. Have you also tried flipping the green direcly in UE4 to see if that is the problem (it looks like that sort of issue)? What is the issue in the first image though? That looks ok to me.

musashidan

Also, don't use .obj if using a synced workflow as it doesn't store bi-normal tangent data. Use .fbx and make sure to uncheck the bi-/t option. That way UE4 computes it on import.

Farfarer

If you're having UE4 compute it on import then you don't want your exported mesh to contain binormal and tangent data.

If you give xNormal a mesh containing those already, it won't calculate new ones for baking.

So OBJ should be fine. I'd still use FBX though... just disable tangent data in your FBX export preferences.

musashidan

Farfarer said:

If you're having UE4 compute it on import then you don't want your exported mesh to contain binormal and tangent data.

That's what I said above: Uncheck.

MmAaXx

I export with those settings here:

Farfarer

musashidan said:

Farfarer said:

If you're having UE4 compute it on import then you don't want your exported mesh to contain binormal and tangent data.

That's what I said above: Uncheck.

My bad, I misread that

Trixx

Hey Guys, I've tried everything here, and none of it seems to work. I am also using a triangulated mesh. I've tried using an FBX, but xNormal generates a blank normal map when I try using an FBX. Unreal imports it just fine. The seam is still as present as ever, with no improvement.

Should I just expect a seam like this? Its not very obvious from a distance; I just want to make sure I'm doing things correctly.

Vailias

In your export settings, be sure to check "triangulate faces" (though you've said you've already triangulated it.. so that might not be the thing) 
Once you have that exported use that to bake. If you're baking data from a mesh that has different normals from your final, you'll have issues.

For a mesh like this don't use a custom cage, just use the ray distance baker included in xnormal.

Trixx

Vailias said:

In your export settings, be sure to check "triangulate faces" (though you've said you've already triangulated it.. so that might not be the thing) 
Once you have that exported use that to bake. If you're baking data from a mesh that has different normals from your final, you'll have issues.

For a mesh like this don't use a custom cage, just use the ray distance baker included in xnormal.

I've exported several times with the triangulate faces checked, just to make sure that it wasn't triangulating elsewhere.

I'm using the exact same mesh to bake my normals as I am importing into Unreal.

I've also tried calculating with just ray distance; no cage. Even with using values from the ray distance calculator in the tools in xNormal, the seams are still present, and ONLY on edges of the UV shells. Every other sharp edge is perfect, so I don't think it's an issue of a bad cage.


Is is it possible that the issue is Blender and not user error? I've been trying to fix this for 3 nights now, with zero success. 

If anyone wants to give it a shot, I'd be more than happy to share my high poly and low poly with you for testing it yourself and seeing if you can resolve it. Perhaps different software such as Maya or Max could yield different results.

musashidan

Post your meshes and I'll take a look

3dsmax/SP/UE4

Trixx

Thanks! I really appreciate it! If you do solve the issue, could you post the steps and settings that you used to make it work? I'm dying to figure it out so I can move on, haha.
https://www.dropbox.com/s/hguzz28mwenrkh6/objtest.obj?dl=0
https://www.dropbox.com/s/yf00uayf7a1vo41/wall_1_exploded.OBJ?dl=0

musashidan

OK @Trixx , I had a look at your meshes. Firstly there was some Geo cleanup.(you had an open border and a few unwelded verts) Then I re-unwrapped your low-poly to a single UV island. This is perfectly fine and actually ideal for a synced workflow as UV/smoothing group splits are unnecessary. Also, you had multiple smoothing groups which I cleared and set to a single group. Note that no cage was necessary and I simply ray-distance baked.
Here is the newly unwrapped UVs:

Next I exported/imported from Max to Substance Painter for the baking. Here are the important settings and the NM:


And here is the bake results:



 

Trixx

Wow, you're a saint! This is perfect! Thank you very much!

musashidan

Trixx said:

Wow, you're a saint! This is perfect! Thank you very much!

No problem mate. I had fun
I would definitely advocate using a synced workflow. It's quite simple once you follow the few basic rules/steps, and cuts out a lot of the problems/issues that usually arise with NM baking and the countless threads here that frustrated members post.

- Wraith -

I had got the same problem one time and I became crazy too. I did this before baking in xNormal and it should fix it ( I made a picture memo to never forget it! ) : It is just for generate a correct normal map. You have to put different smoothing groups for each UVs Shell, and you can export this mesh for xNormal. After the baking done you can apply your usual smoothing groups (auto or 1) on your lowpoly mesh you want to integrate in the engine.