Normal Maps 101 - Help needed
Good day!
I have looked through the sticky threads and while they seem to be ripe with technical info, I am so much looking for ways to make the understanding of that info more tangible and accessible as I just can't seem to grasp/apply it.
I can't even get the easiest normal map baking to work - cube with rounded edges to 6-sided cube (hard edges, lowpoly).
I'm working in Modo 902.
Situation:
Rounded cube in wireframe encapsulates the standard cube.
http://i.imgur.com/oVu1JAK
UV-Map of standard cube
http://imgur.com/zMfUQKN
Applied normal map (lowpoly material has 180° smoothing).
http://imgur.com/aQpfQdq
Said normal map (looks horrible)
http://imgur.com/mHmDfJz
Applied normal map (lowpoly material has 89° smoothing).
http://imgur.com/zMfUQKN
Said normal map (looks better, but doesn't give wanted result).
http://imgur.com/gHKZ8zP
The baking has been done without a cage, distance set to 5cm.
How do I choose the correct distance?
Should the lowpoly be inside or outside or as close as possible/intersecting with the highpoly?
When should I use a cage?
With the material's smoothing cranked up to 180° surfaces that should be totally flat get a gradient in the normal map = change of normal, right?
This shouldn't be, what to do?
I ask for forgiveness as I guess this has been asked so many times before. I could not find the answers I'm looking for and hope for a good soul to help me out.
Maybe I'll grasp the technique one day so I can make a guide for idiots like myself.
Thanks for reading.
Replies
1 - in this workflow(more modern) a single smoothing group is used for the entire mesh, both the baking software and the destination software must both use the same tangent basis(i.e - Mikktspace), UV/SGS splits don't matter as there is only a single SG(splits at UV borders are optional)
2 - in this workflow(older) you must have a change in SG anywhere that hard edges meet on the mesh. If there isn't a break in SG the edge will be soft. Anywhere you have a change in SG you must have a UV border split.
This is it at its most fundamental. For more in-depth explanations from the pros on this site I recommend studying the stickies, as every bit of info you need to bake perfect maps is in there.
Cage distance can be measured through test-baking. But very importantly, the more precisely your low-poly matches your high, the better your bake result.
Cage usage will vary mesh to mesh. This can be assessed when you understand what, exactly, the cage is doing. For simple meshes without acute-angled concave areas, or harsh 90 degree corners with no supporting geo, usually distance ray-casting is enough.
The gradient in the NM is the result of the baking process compensating for extreme surface changes with no SG breaks. In a synced workflow(#1 above) this isn't a problem, per say, but can lead to issues with compression, mip-mapping, and LODing.
Again, I can't stress enough the wealth of info in the stickies here. Rather than just browsing I suggest vigorous study in conjunction with vigorous practical testing.
I know it seems overwhelming at first, but it really isn't once you understand a few simple rules to follow.