Red channel gets inverted along seams in normal map

Ruflse

Hello everyone, I have a little problem with my baking.
I'm trying to bake this model into 3 texture sets but for the normal map of every one of them the red channel gets inverted along the UV seams.



I've had some troubles with seams before but this is the first time this happens and I can't think of a solution. Each side of the body, jacket and shirt have their own space in the UV map, so they are not overlapping. The bakes I'm showing are made in Painter, but I have tried baking in Blender to see if the problem is the export of the lowpoly and I have the same errors there. Every piece is one smoothing group since Blender doesn't have that utility unless you split the pieces, and I'm not using a cage.




This causes some really ugly shading inside Marmoset too so I'd like to fix it. Any ideas?

frmdbl

You don't need to split meshes in Blender to have sharp edges,
just mark them as such and use the 'Edge Split' modifier or the 'Autosmooth' property.

Ditch the concept of smoothing groups, it's just an overcomplicated way of assigning sharp edges (or normal splits)

Such mismatch at the seams doesn't necessarily mean the bake is incorrect (as it's tangent space)
What's sort of ugly shading do you have?

Ruflse

Sorry for the late response. I solved part of the problem in Marmoset, I had the normal map affecting the diffuse as a layer in Painter so the seam appeared in the diffuse too and forgot to flip the green channel in Marmoset and to check Scale & Bias. I don't have problems with the shading anymore even if the seams can still be seen a bit.
I never had this kind of bake before tho, why does the red channel get flipped along every seam? I checked the normals and I don't see anything weird with them.

Edit: Nevermind, I found the answer in Polycount's wiki 
wiki.polycount.com/wiki/Normal_Map_Technical_Details

ferverduchp

Hi @Ruflse, it's been a while since you solved this problem, but I have now the same problem you had, and I haven't found any other place where this issue was talked about, could you tell me how did you solve this problem? Thanks! 

Btw, I'm gonna leave here some screenshots of my issue to make you easier to understand. 

Ghogiel

The OP mentions he had the TS normal affecting a diffuse layer, so even if the shading is correct in the normal, the color difference along seams in normals would still be visable in the color map.  And aparently his G channel was inverted, which would cause a seam. He also mentions he found a solution here, http://wiki.polycount.com/wiki/Normal_Map_Technical_Details , to his R channel being inverted (which you can just do manually in photoshop or in a tick box if it is indeed inverted (I am sceptical of that being the issue, I can't think of a renderer that uses inverted red channels in the normal, and probably the only way to bork it in the bake would be inverted face normals on the high or low, which would mess up a lot more than just the R channel)

Bohemian_cz

Hello there, I have same problem but tried everything you are writing here and still have this problem. Baking in marmoset. Fliping G chanell doesnt work, tried fbx, obj. Have harden edges along seam borders, Normals on low poly are correct and low poly model has support loops as well. Still have flipped tangents along uv seams borders. Some people tell rotating uv island helps, but not tried it as well. Problem is that the model is spherical as you can see on uvs.

pior

Well, if "some people" give you advice ... then please quote said advice, as it is either wrong, or badly interpreted :)

Tangent space normalmaps are relative to both the surface information of the high and the UV orientation of the islands of the low. It is perfectly normal (!) for color changes to appear at seams, this is math (assuming that your UVs are clean of course). On top of that Toolbag has some options to compensate mirrored shading if I am not mistaken, but that's besides the point here.

To test this out intuitively, create a very simple test case with a simple smoothed cylinder as high, and make a few variants of it with different UV orientations. Then post your results here.

Neox

tangentspace normalmaps appear to be human readable but they are not. you can do some tinkering with them but what you are expecting is simply not possible with these maps. as Pior said, they are based on the lowpoly geometry, their tangents and UV orientation, previewing them as if they are a diffuse map will only lead to confusion.

but on a side note, i would suggest you straighten your UVs, even with techs like nanite around the corner this is more important than ever,