Ambient Occlusion map has a grid pattern
I am sort of in a predicament between bad normal map but good AO or good AO but bad normal map.
I either make my "high bake object" lower poly and my AO map has terrible lines and when I increase the poly count of my "high bake object", the bad AO map lines go away but now the beveled edges from the normal map look terrible.
I generally have allot of issues with baking edges into normal maps and have found that subD often is bad. The reason being it makes the high edge be at a different position because of the subD smoothing. So I try to when possible have the high poly not have to many subD and more so things like bevels.
But now I am noticing that this low detail "high bake object" is making my AO maps look bad. And I am quite sure it is because of topology because this ugly grid in the AO map looks just like the topology.
In this image you can see what I described:
Here is a close up from the bad AO map
It most definitely has nothing to do with samples, ray count, image bit depth, because I cranked them up insanely high to see if it would fix the problem. Cage and normals are also fine .
I am using Marmoset Toolbag 4.03 to bake
(subD = subdivision surface modifier)
Replies
My temporary solution is going to be: do 2 AO map bakes and in PS mask out those bad surfaces and replace it with the other AO map
You don't have enough geo in the high poly, simple as that. :)
SubD modelling is a big topic, but you control the smoothing with support loops and creases to get the smoothing where you want it to be.
I'm moving your Topic from General Discussion to Technical Talk.
There's a Sticky in the latter section that will help you with highpoly modeling: How The F*#% Do I Model This? - Reply for help with specific shapes - (Post attempt before asking)
There are also a bunch of Topics on normal mapping in the Sticky: Long-Running Technical Talk Threads
Support loops help keep subdivisions in check to some extent but will not do anything in this situation.
The only way I can think of combating this, is to just not use subDs for pieces like this. It is very frustrating but I just can't find another way. If you have a suggestion I would love to hear it
You might find this useful (from Long-Running Technical Talk Threads)
Understanding averaged normals and ray projection/Who put waviness in my normal map?
Thanks, ill be going through all those sticky posts, adding loops on the low to support the shading is definitely something I did not know.
Add more geo to your low poly so the curve of the surfaces match more closely :)