Baking sharp deepening problem

bicuk

Hello!
Can someone explain how should I solve this common baking problem with sharp deepening? 
I mean sharp deepening with ray distance produce bad result in these deepening 


is there only one solution - just spend a lot of time to make properly cage model? (nope, i do not want to do that)
I think it is not a blender problem, because marmoset do this similarly. (or does marmoset has some functionality for problems like that)

gnoop

I hate cages too.  And I have not noticed marmoset  does it any different.  But I am not sure, haven't used it much.

I myself just have all vertex normals up in such case. it helps  with lods too. And I keep same UV on hires model.

Then I just  have a material in Blender where input geometry  node normals  have been *0,5+0,5,  for each of RGB.  It make them coming from -1 to 1  into  0-1 positive range.    Then I apply it  to hires object   "emit" slot  and  bake emit  slot.  Now I get classic object space normal map  and save it as 32 bit exr,      

Then I apply  this object space normal map texture  as  normals  for low res object having same UV   and bake normal slot as  tangent normal map.  It works with whatever splits or hard edges or triangulation your lowpoly model  has.   You can even remodel it later if UV stay same or re-project UV back with data transfer. It just needs UVs on hires model to bake object space normal map on hi res first.

The method is ancient like  pyramids,       it's based on a fact that object space normal colors are not related to triangulization or vertex normals   and if you save the obj space bake as 32bit exr   it usually have enough precision for re-baking it into 8 bit tangent space texture later .      Just apply rotation  before baking object space.

Ghogiel

In marmoset you can paint the cage distance pretty quickly in those tight spaces and get realtime updates on how it's looking.

If your low poly is properly covering the high poly well in the first place your initial offset distance can just be so low that you will get all the high poly under the mesh before you get cage intersection errors.