Xnormal problems - use cage only projects outward?

Ghostscape

I have a rather dense mesh that isn't baking cleanly through 3ds Max and isn't easily disassembled, so I figured I'd take it into xnormal. Xnormal is ridiculous fast, but I'm getting massive errors.

I'm exporting with a cage from Max, since there are parts where I need a long raycast and others where I need a much shorter one.

After turning off "use nearest on miss" which was resulting in significant portions of my mesh casting onto the exploded parts of the model and missing the parts they were supposed to catch entirely, I realized that every concave part of the highpoly mesh is being missed. I have a number of indentations in the high poly that have a flat low-poly above it, and each one of those areas is missing it's raycast.

Is there some setting I'm missing? The cage appears to be working nicely to catch the outward raycasts, but the inward ray casts don't appear to be happening at all.

jogshy

Ghostscape wrote: »

... I'm getting massive errors.

I'm exporting with a cage from Max, since there are parts where I need a long raycast and others where I need a much shorter one.

Can you post some screenshots with your settings, normal map and cage setup in xNormal, pls?

And... did you see any error/warning while exporting the max's cage using the SBM exporter?

After turning off "use nearest on miss" which was resulting in significant portions of my mesh casting onto the exploded parts of the model and missing the parts they were supposed to catch entirely,

Do you mean "Use closest hit if ray fails" option? In theory that should be always checked to avoid errors if the ray fails.... but can be a problem for some complex models... you can try to disable it and see if solved the problem.

I realized that every concave part of the highpoly mesh is being missed. I have a number of indentations in the high poly that have a flat low-poly above it, and each one of those areas is missing it's raycast.

Perhaps you should try to enable (or disable) the "Discard backface hits" option in xNormal... and sure the highpoly model's normal are well exported(enable the show tangent basis + show normals option in the 3D viewer)... it's not the first time I see the 3dsmax2obj exporter to mess completely the vertex normals... Use better the gw::Obj exporter if you need an .OBJ or, even better, the xNormal's SBM mesh exporter.... and be sure to apply a Reset XForm + Edit mesh(to triangulate) + collapse if possible.

Also sure the "use cage" option is checked on the corresponding lowpoly mesh slot ... if not, the uniform ray distances will be used...

Oh btw... if your model's radius is very small(<=1.0) you can try also to scale the mesh a bit up... due to floating point errors the small numbers can can problems... so scaling the mesh a bit up can help to avoid ray fails.

I hope it helps.

Ghostscape

I can't post the actual model/maps, but I've got a shallow indent in the highpoly, with the low poly lying flat above it. The indent is showing as a missed raycast on the wireframe/ray failure map. So it looks like the rays are cast out from the low poly to the cage, and rendering correctly, but it never goes inward to find the low poly. If I have "use closest hit if ray fails" checked it doesn't look inward to the closer mesh, but instead looks meters away (I have the highpoly/lowpoly exploded to avoid intersections)

I'm using xnormal to generate averaged normals for the high poly. It's exported as an SBM

my low poly has an edit mesh on top of it, and then a projection modifier. I'm exporting the cage, normals, and UVs on my low poly.

use cage is checked.

If I use an external cage Xnormal crashes to desktop. Could it be that my low poly/cage is busted?

I tried scaling both meshes by 100 in Xnormal with no change.

jogshy

Ghostscape wrote: »

I can't post the actual model/maps

Roger. Well... without that info gonna be hard to figure what's happening... Ideally you could send me the models so I could debug it(I treat all the models confidentially)... but if you can't I understand it.

The indent is showing as a missed raycast on the wireframe/ray failure map.

In red color, right? Interesting.

So it looks like the rays are cast out from the low poly to the cage, and rendering correctly, but it never goes inward to find the low poly. If I have "use closest hit if ray fails" checked it doesn't look inward to the closer mesh, but instead looks meters away (I have the highpoly/lowpoly exploded to avoid intersections)

And If you check/uncheck the "discard backface hits" option? If you use that option and your model normals are inverted then the rays can fail if they cannot hit other parts in the mesh.

I assume you aligned well both meshes... did you see them well aligned in the 3D viewer? Have you tried to show the tangent basis in the 3D viewer? Do you see the blue axis(normals) pointing to you or to inside? Are you sure the cage is covering completely the highpoly mesh?

I'm using xnormal to generate averaged normals for the high poly. It's exported as an SBM

That's right... but, what happens if you export it as an .OBJ and you extrude a bit the cage in the 3d viewer?

my low poly has an edit mesh on top of it, and then a projection modifier. I'm exporting the cage, normals, and UVs on my low poly.

That's ok but... have your ResetXForm it + collapse? ResetXform is critical, specially if you rotated your model.

use cage is checked.

Ok, that's fine.

I tried scaling both meshes by 100 in Xnormal with no change.

Well, that can discard any scaling problem if the mesh is very small... but if it's too large then perhaps you can try to scale it down(x0.01 for example)... what's your model radius btw? And what xNormal version?