Help - Maya normal map transfer weird behaviour/seams

Zodd

Hello everyone,

So i've ran into this issue and i cant seem to find a way to get rid of it. What happens is a weird seam shows up on my normal map edge and i cant find a logical explanation for why its happening, from what i can tell the maya transfer map engine misses a couple of pixels and then "finds" the edge its projecting again so this makes a gap in normal map channels, this is most visible when dilatation/fill texture seams is added so it makes these trapezoid shapes.
Tried xnormal and it does not make there "dents" in normal map, unfortunately it does make full seam along the edge so theres that..


So why is this happening ? Is it something that is an issue with maya-s transfer map option? since again,  this does not happen in xnormal (missed rays). I mean i would understand if i was trying to bake some funny 1 degree angles and the normals were coming out messed up but this is regular run of the mill 90 deg. angle and it returns this errors.
Also the beginning of the missed pixels start at the vertex but disappear halfway through? is this normal(sigh) behavior ?

And here is what ive tried so far:
Increasing texture resolution to 4096^2 - seam still visible
Making new geometry lp and hp - seam still visible (same places)
Manual Envelope/Cage editing - seam still visible
If i stitch UV's together the seam dissapears but does not solve the issue since i have to have split UV's somewhere
Hard/Soft edge - still there
Cranking up the "sampling quality" "filter size/quality" - still there and visible

huffer

Have you tried straightening the UV shell, so all edges are perfectly horizontal or vertical?

Zodd

Forgot to mention, yes i did, on every edge that can be straighten i did align the UVs,
Only other thing that comes to mind is there is not enough resolution reserved for this part, however since ive tried baking these out on a 1024^2 map
and the problem persists... its not that?

And also this is from Maya2016 is there maybe some bugs related to its way of doing normals specifically?

huffer

One other thing it might be is tiny tiny mismatch between high and low, maybe try to push the low poly futher out and make sure every loop is straight on both geometries... If not ... #justmayathings - use xNormal with cage and you shouldn't have any seams.

Zodd

Hey thanks for the reply again, yeah i kinda switched to xnormal now i do have seam(s) along entire edge(s) but they are because of the way maya interprets normals made other software... at least thats the explanation i got from diggin around google for the issue.
Here is how it looks like now:


There are still some sharp edges visible but much less than when maya was used to bake the normals.

I dont know if i mentioned but i did try and make totally new high and low poly (segments only) then unwrap low poly the same way i did in the original and then do a bake, and yup you've guessed it got same weird half seams on same spots, that led me to believe that it must be one of those "justmayathings" moments i stumbled upon.