low poly vert normals impact on normal map baking with cage in XNormal

ojkoorde

I've been searching for a few days to the answer and still cannot find it. Would be great if someone clarifies it to me. According to all logical sources and how I understand cage baking - the cage is there to:

1. limit ray distance
2. determine projection ray direction

Assuming cage is the same topology - rays are projected in direction determined by every pair of two points with same UVs between cage and low poly and only 'ray with hipoly' intersections between lowpoly and cage are considered.

According to this - neither cage nor low poly vert normals are meaningful and yet xNormal's output changes if you change low poly vert normals (and this is the only change ie. do soem cage bake, change some vert normal of low poly, bake again and you get different result). I've tested Marmoset Toolbag and it ignores both cage and low poly normals as I would expect it to do.

Why do low poly normals matter for xNormal when baking with cage? It does not make any sense to me.
Am I missing something here? Is there any purpose for that behavior in xNormal?

xNormal's manual is misleading saying that 'a cage is just a manual designed extrusion to specify max ray cast distance'.

ZacD

Low poly vert normals are meaningful, that's why they are called tangent space normal maps, the surface normals of the model directly effect the bake. 

Artists mostly use cages to avoid projection errors, such as skewing from projection errors or preventing the baker from looking at an overlapping piece of geometry instead of the intended part

poopipe

In a perfectly synched workflow the normals on your low res don't matter - you can simply average everything and it all looks perfect. 

The changes in your normal map output are  xnormal compensating for the changes in mesh normals. 

if you apply the same map to two meshes with different normals then things go wrong

Farfarer

The normals and the UVs of the low poly mesh form part of the "encoding" for the resulting baked normal map.

Changing either of those makes the "decoding" of the normal map appear incorrect.

EarthQuake

poopipe said:

In a perfectly synched workflow the normals on your low res don't matter -

Wrong, they matter, just not as much as in a non-synced workflow, where massaging your mesh normals can be very important.

you can simply average everything and it all looks perfect. 

Also wrong. This works in certain cases (mostly when you have very dense meshes without extreme normal direction changes), but there are benefits to paying attention to mesh normals with a synced normal workflow. Using hard edges in certain areas for hard surface work means the normal map has to do less "heavy lifting", so it will compress and mip better. Using hard edges can also help to prevent "resolution based smoothing errors", which happen when your normal map can theoretically compensate for the mesh normals, but you don't have the resolution to actually do it, which results in streaky lines in the shading.

Here's an example:



More general information about how normal maps work can be found here: https://www.marmoset.co/posts/toolbag-baking-tutorial/

The changes in your normal map output are  xnormal compensating for the changes in mesh normals. 

if you apply the same map to two meshes with different normals then things go wrong

Correct.

poopipe

Fair points..  I was over-simplifying cos I couldn't be arsed with the caveats. 

ojkoorde

thank you guys