Normal map seams everywhere except maya?

Zodd

Hello everyone,

So i've come across this weird problem that i get seams in my normal map regardless of what i do, happens with almost every software except maya ???

Encountered this behavior on part of my mesh, could not fix it and then went back to basics with a cube

Here is my setup for low poly:

>Left one is HP right is LP encompassing HP, plus vtx normals/hard edges

>Standard hard edges on separated uv islands

>Triangulated, LP is same as HP only things sticking out of hp are 90 degree edges

>Have cage (can be used in xnormal and maya)

Below is a huuuge image displaying the problem in substance painter 2 

Regardless of the tangent spaces and stuff there is always a seam, now i know i shouldn't be too picky and there will always be seams in normal maps. However i kiiinda don't think they should be this obvious, since the SP and marmoset give worst result

Plus normal map from maya is 512 and almost no seams, unless i go in really close with camera, and even then it holds up satisfactory.

So if someone can give me some sort of explanation i would be forever grateful, since i doubt that literally only maya can give satisfactory results while other programs that are kinda industry standards can not, also interned is full of people making/baking good normals in the aforementioned programs...  must be me missing/overlooking something...

Thnx in advance you guys

gvii

It might be a color space issue. New versions of maya have a color space setting in the file nodes for the texture maps --- maybe it's different color spaces for the texture generated across the programs. Try baking in substance painter then using that texture in maya --- you will probably have to change the color space in maya's normal map file node to "Raw".

ant1fact

what does your cage look like? your cage shouldn't have split edges

Zodd

Hey, thnx for the replies

So there may be some more explaining to do,

What i'm trying to achieve >

Make high and low in maya and do the materials in SubstancePainter, so i wanted to bake all of the needed maps in SP (including normal) and present it in marmoset, all of the programs use different tangent space BUT they do have option to switch to other softwares tangent space, meaning its all cool, if i manage to get normals from substance ill be able to switch to those in marmoset and should be cool.

Trouble is at the moment everything (except maya) makes seams, i figure there will also be errors in other maps (curvature, thickness etc..)

So in general mayas normal map works everywhere and is ok, but normal map from other bakers have seams on edges that are 90 degree and have uv splits + hard edges.

@ant1fact Regarding cages, maya automatically makes a cage (envelope) that is identical to the LP and in xNormal you have to have exactly the same mesh, with all the vertex numbers corresponding to the LP perfectly so it has hard edges but the geometry is welded, as for using cage in SP i haven't really seen any difference between using it and not using it

 @gvii tried switching to maya and using different color space and no dice, seams are exactly the same, so same position same intensity etc..

Also its worth mentioning that if i bake at 2K its kiiinda ok, so there are still seams but they are not as visible, and i kinda think it defeats the purpose... its either ok at 512/1024 at reasonable texel density or it isn't at any right ?

Also thnx

poopipe

Try pixel snapping your uvs

Obscura

Do you have dilation?

Zodd

@poopipe Did not try pixel snapping, just tried and the problem persists, marginally better than the above screenshot for SP2 bake

Like so:

@Obscura Tried various dilatations from 4 to 16 to maximum, still the same result, and in general i tend to always put at least 4-8px,

Something i just remember reading somewhere is that maya does some post render smoothing of pixels, so the results may be "skewed" in its favor, but  regardless people have baked non seam normal maps in other programs... its doable is what im saying i just seem to be failing at doing so for some reason.

Any baker should be able to put this type of detail in normal map without seams this visible right?

pior

I think you are (unintentionally) operating on a logical fallacy.

"interned is full of people making/baking good normals in the aforementioned programs...  must be me missing/overlooking something..."

Fair enough, but these people also probably put all the odds in their favor by using best possible practices. Now of course trying to get down to the bottom of an issue by using a worst case scenario can be valuable ; but in your case, you are currently doing this :



In other words : you are feeding the harshest data possible (a hard edge, with completely discontinued vector data across said edge, encoded into a low res RGB texture) into a system that is best used to handle very minor, tertiary changes in surface quality.

In a way what Maya does or doesn't do to make things appear smooth is almost irrelevant. Sure enough, understanding what it does math-wise can help you find out issues and understand how the whole thing works math-wise (especially when working with an engine that has been precisely designed to be 100% working with Maya data) ; and as said, working with worst case scenarios is a good thing to do as far as research is concerned. But if your goal is to create assets that look just as smooth/good as what you see on awesome portfolios online, then you should start by using the least fragile setups - as in, not putting hard edges and UV splits around a model without a reason.

Basically what I am trying to get at is, while there is value in understanding what's going on here, knowing this will not necessarily help you in your bigger goal.

I hope this makes sense

Zodd

Hey-a

@pior Yeah, i get what you are saying, and i kinda managed to get good results on the piece im actually working on (not a cube   ) by adding chamfers to the edges, it kinda equalizes the score of 90 degree edge and a UV split vs 2 edges and no UV split (not exactly but kinda)

Also i think i understand what you mean by baking across discontinued vector and the results do seem like its some sort of a precision error, what caused me to have an issue with it was, error is not in every baker i use (only in 3/4 )

>and>

Bake from maya looked identical to what i was seeing in it, even tho the png was just jammed into the other programs (no adapting vert normals to mikkt or other tangent space) just load it up and no seams.

>and>

People seem to be able to pull it off, again concerning this one i totally understand your answer about pulling every odd in my favor

So it seemed "easy" plus i thought i understood better what goes on behind the scenes when baking normal maps from the sticky tech talk threads << And regarding this if anyone has something i can go and read upon, pls share.

Now, it seems the links are down in the standard threads that i make pilgrimage to, each time i come across this type of problem (understanding normals and ray projection, making sense of skewed normal map details, bit depth and banding) so i cant show you the thing im referring to when talking about "baking soft edge on hard edge" but you get the idea.

Regardless, thanks for the answer(s) and for the end here are some new bakes with less fragile setup

Turned out satisfactory seeing as its 1024

and the wires

So thank you all for your input and hopefully next thread i start is going to be in waywo

Assassin115

@Zodd Hi! I just wanted to know if you found a solution for this cube issue? I’m currently facing the same problem, these weird edges are just visible regardless of the settings that I apply in SP so I would be really glad if you could help me with that

gnoop

It's never perfect on split edges,  should look juts acceptable enough from some distance.   Also it's useless to  make choices based  by how 3d soft shows you that . Check in game  with shadows  and all lighting features.   it might be even worse with low res shadow maps or some indirect illumination techniques. 

A safe approach is to make geometry bevels with smooth shading  and face weighted normals  where it's visually important  and keep split/hard edges in small things .  But it's also depends on how close your next lod is supposed to appear .    I personally bake normal maps for lod_02 usually.

Neox

what does your HP look like inside marmoset? does it have the same UVs as the highpoly? chances are you have hard edges on those UV seams, despite what your HP looks like in Maya or zBrush or the like.

baked from HP without UVs

baked from HP with UVs, its the same Mesh just onece with and once without UVs

that being said, yeah mayas baking despite really really slow is pretty darn good. especially with its sharp edge smoothing/blurring you can turn on. but its so damn slow that productions tend not to use it anymore and trust marmoset or painter as you usually do not get close enough to spot the issue. also with compression of the maps in engine, and possible compression of the UVs, if you get close enough you will always find something.

Assassin115

@Neox I didn’t quite understand what you mean by “HP (high poly?)” and “does it has the same uvs as high poly?”. I do not unwrap high poly, only low. I also don’t use marmoset, I do the baking in substance painter. It seems that it is maximum that you can get from this low poly count and if you look from a distance, seams are not really visible

Neox

HP is highpoly yes, i didnt test in substance painter but as you showed marmoset i assumed you also baked there.

As i made the highpoly out of my lowpoly it had UVs, often times its not good to have UVs on the Highpoly because it can result in hard edges inside marmoset.

Assassin115

@Neox Oh, alright, I duplicate my low poly before UV unwrap so high poly would not have any UVs. Guess I'll try marmoset or Xnormal then cause they tend to give better baking results, at least this is what I've read from forums...

Noors

Bakers are still doing a not so great job at padding normal maps.by extending the color (so the same direction) instead of estimating the slope.
https://shaderbits.com/blog/uv-dilation

Noors

Neox said:

HP is highpoly yes, i didnt test in substance painter but as you showed marmoset i assumed you also baked there.

As i made the highpoly out of my lowpoly it had UVs, often times its not good to have UVs on the Highpoly because it can result in hard edges inside marmoset.

I trust you on this, but that's the first time i see HP uv having any influence on the bake. Might be some toolbag voodoo, it's really weird.

Maybe the extra points caused by uv cuts arent interpreted correctly.

Neox

Noors said:

Neox said:

HP is highpoly yes, i didnt test in substance painter but as you showed marmoset i assumed you also baked there.

As i made the highpoly out of my lowpoly it had UVs, often times its not good to have UVs on the Highpoly because it can result in hard edges inside marmoset.

I trust you on this, but that's the first time i see HP uv having any influence on the bake. Might be some toolbag voodoo, it's really weird.

Maybe the extra points caused by uv cuts arent interpreted correctly.

happens all the time on meshes coming from zbrush with crude UVs going on. we have export settings for our HPs which turn off export texture coordinates for all meshes before export, just to avoid these hard edges.

pior

Noors said:

Bakers are still doing a not so great job at padding normal maps.by extending the color (so the same direction) instead of estimating the slope.
https://shaderbits.com/blog/uv-dilation

That wouldn't work though, because the color doesn't only depend on the slope but also on the UV orientation. Because of that, there is no other better color than that of the edge to blend towards, because the actual color "on the other side" can be completely different (even on a very shallow slope). Blending towards that would only make shading seams even worse ...

Noors

pior said:

That wouldn't work though, because the color doesn't only depend on the slope but also on the UV orientation. Because of that, there is no other better color than that of the edge to blend towards, because the actual color "on the other side" can be completely different (even on a very shallow slope). Blending towards that would only make shading seams even worse ...

I'm talking about estimating the direction of padded pixels in the correct tangent space, not blending to the other side edge color. It doesn't even have to be necesseraly estimated as it could be known.
No ?

gnoop

It is nothing  comparing to issues a game engine would cause.    I recall U4 made this hard edges instantly revealing after you bake  lightmass indirect illumination  or whatever it been called there.   Some SSAO techiniques also  make those seams visible.  Low resolution shadow maps  also instantly reveals that "rounded" hard edge when you look at it from shadow side. 

Comparing to all those issues this subtle seam  you see only up close  is really nothing to worry about.   You usually have  some color, roughness, metallic  textures of a noisy nature that help to hide it too.   BTW it's kind of less visible on shiny metallic materials.

You can always get rid of  it with beveled geometry and face weighted vertex normals.  In fact it would  be even more effective calculation wise.  Less vertexes to calculate because of tri-striping and less UV seams.

The nice side   of hard edges is that  they make your vertex normals stay same through different lods and  kind of  fool safe when  somebody accidentally  messing  vertex normals with  random mesh editing.

pior

No ?

Well, maybe, but I'd be very curious to see any case where such a convoluted post-process (involving somehow fetching the normals information from the "other side" of an edge, converting back to the proper tangent space of the initial side, and feeding that into the padding of an island edge) would have any impact. All that for merely ... one pixel ?

It sounds to me like something that would only be somewhat useful in the case of having both : hard edges in the middle of a continuous smooth surface *and* rays being cast split over that edge - which are two things that shouldn't be done in the first place anyways. And even there, it would require to manipulate the pixels on the "right" side of the edge too, for a smooth transition. So I guess I really don't see when that could possibly work  

It's a bit of a common topic when it comes to normal mapping/baking issues : people often trying to develop complicated workarounds, whereas the issue is actually being caused by bad practices.

Noors

Yes it would be for 1 pixel, but what a pixel.

One day it will be the standard workflow and i'll come back to you

marenin

Just want to bring up this related old thread
https://polycount.com/discussion/149028/padding-issue

Noors

marenin said:

Just want to bring up this related old thread
https://polycount.com/discussion/149028/padding-issue

Oh i forgot about that. So it's indeed technically possible. Way to go !

And even if stuff can hide those artefacts, or if some workflows ignoring normal maps will create artefacts anyway (eg good old lightmaps) i dont think it's a reason to not limit them at the source level.
Or else we could have kept on with broken tangent space workflows, "dont worry, just put dirt on the texture, it will hide it"  (Yes i've seen that.)

Eventhough we will soon import billions tris HP straight into the engine and no more normal maps issues ^^

marenin

It is possible, yes.
Actualy in my example in that old thread baking process get hipoly information beyond uv shells, so it real geometry data, instead of tricky padding that just "hide those artefacts".

..broken tangent space workflows..

"dont worry, just put dirt on the texture, it will hide it"

Eeew ..

pior

@Noors , @marenin : these examples (with the padded tube) look great ! And while that does look much more involved than a 2D post process, well,  if it works reliably and saves time and headaches then it is worth it  

Just to make sure : was that tubular mesh sporting hard edges in the middle of the cylinders, and/or a split cage at the location of the seams ? In any case the processing does look useful, but I am just curious as to what the starting point actually was.

It would certainly be interesting to see this make it's way to commonly used tools .Blender ? Toolbag ?

marenin

was that tubular mesh sporting hard edges in the middle of the cylinders, and/or a split cage at the location of the seams ?

No hardedges, no splits. Just two regular intersecting cylinders.

pior

Hi there @marenin , thank you for the details. In that case I am somewhat surprised that the regular bake (without the special post process) showed such amount of visible edges. Perhaps the rays/cage were not thrown along the normals (since if I am not mistaken, some baking apps do split the rays at UV seams) ?

By all means the result with the special post process does look great, flawless even ; but I find it odd to see such breaks in continuity in the initial raw bake. Of course it is a bit hard to tell from just pictures as there are so many factors involved, as always.

EarthQuake

Neox said:

Noors said:

Neox said:

HP is highpoly yes, i didnt test in substance painter but as you showed marmoset i assumed you also baked there.

As i made the highpoly out of my lowpoly it had UVs, often times its not good to have UVs on the Highpoly because it can result in hard edges inside marmoset.

I trust you on this, but that's the first time i see HP uv having any influence on the bake. Might be some toolbag voodoo, it's really weird.

Maybe the extra points caused by uv cuts arent interpreted correctly.

happens all the time on meshes coming from zbrush with crude UVs going on. we have export settings for our HPs which turn off export texture coordinates for all meshes before export, just to avoid these hard edges.

Can you get us some test files for this? This is not a problem that we're aware of, and if it reproduces we can probably fix it.

Neox

where can i reach you nowadays? discord?

EarthQuake

Neox said:

where can i reach you nowadays? discord?

support@marmoset.co is good for bug reports, but yes we have a discord too https://discord.gg/5YsseU4Cx2

Valvoa

gnoop said:

A safe approach is to make geometry bevels with smooth shading and face weighted normals where it's visually important and keep split/hard edges in small things . 

Hi gnoop, when using this technique can unwrapping be more problematic? Say I have a cube with a 1 segment beveled edge and face weight normals applied. I won't be using a normal maps to smooth out the edges as my edges are already smoothed but now. Just wondering how artists would approach this on a more technical midpoly model as I'm experiencing this currently. 

gnoop

Justin Voye said:

gnoop said:

A safe approach is to make geometry bevels with smooth shading and face weighted normals where it's visually important and keep split/hard edges in small things . 

Hi gnoop, when using this technique can unwrapping be more problematic? Say I have a cube with a 1 segment beveled edge and face weight normals applied. I won't be using a normal maps to smooth out the edges as my edges are already smoothed but now. Just wondering how artists would approach this on a more technical midpoly model as I'm experiencing this currently. 

On beveled  edges you can have no uv splits   That's all about Unwrapping.