Max to Xnormal bake issue

damageINC

Hi all. Been a good long while since I posted in any forum but I've hit a snag and I was wondering if there was a solution. I'm modeling in max 2014 and baking in Xnormal solely using ray distance values (not using a baking cage).

My issue is that for whatever reason, when I bake with a mesh comprised of quads, I get a number of faces that display what looks like a shading error across them (looks like an X shaped shadow across an individual quad face). If, however, I export as an FBX with triangulation the errors basically disappear. I have tried using Turn to Poly and setting the size limit to 4 to force the mesh to maintain quads even if I toggle the FBX exporter to hold on to edge orientation and it still has bake errors.

Normally I'd just triangulate it but if I want to go back and edit the mesh or have it rigged I assume triangulating it will break vert order.....so I would prefer to keep it quads if possible. It's just easier to work with as an editable poly mesh quite frankly. Any ideas? I've used this workflow for a good long while and never seen this before.

Bek

This is normal. Surface shading (which a TS normal map relies upon) can change on triangulation — since there are multiple ways a n-gon or quad can be triangulated, you can have a mismatch, hence the X-shaped shading error. You ALWAYS want to bake with the same triangulation as will end up on the final mesh. There are two ways you can achieve this, the simplest being triangulation before you export (but, obviously, on a duplicate — you don't work with a triangulated mesh, ever). You might also be able to configure xnormal (look under settings/plugins) to triangulate in the same way as your 3d package.

I assume in max triangulation can be applied in a non-destructive way via a modifier, which should save you having to duplicate your mesh, trianguate it, then export it. You can possibly also enable triangulation on export — but I don't use max so I can't be certain.

damageINC

@BEK: Thanks for the advice. Yeah I did a ton of experiments with this. Some like you suggested actually, where I took two copies of the mesh. One with triangulation set as my LP bake mesh......and a sister version of the mesh that retained its quads in max. Bringing the quaded mesh into Marmoset2 and applying the TS Normal map I baked from the triangulated version didn't alleviate the issue. I need to do some further tests, but none thus far have been successful.

Bek

That's because a quad mesh in Toolbag 2 is still triangulated (behind the scenes) and the way TB2 triangulates might not necessarily be the same as the way max (or xnormal if you fed it quads) triangulated. It's not just the information you're baking to the map that is important, but the information on the mesh itself (surface shading) as they interact. If you were to bake a map that did not care about lowpoly surface shading (like an object space normal map) you will not notice this error. So, the triangulation you bake with MUST match the triangulation you end up using the baked map on. So work with quads, send the same triangulated mesh everywhere else.

Edit Quads. Display Tris.

damageINC

Makes sense. Thanks.   : )