Home Technical Talk

Triangulating in 3ds max before baking?

Hello, so from my minimum noob understanding I supposedly have triangulate the model before baking it, otherwise the game engine might triangulate it in ways I wouldn't want it to. However when I use the "turn to poly" modifier in 3dmax to triangulate the mesh it completely messes up my smooth groups, consequently messing up my bakes as well... Is there any way to deal with this? Please, help. I don't wanna have to do all the smooth groups over again...

Thank you very much in advance!!

Replies

  • Ghogiel
    Offline / Send Message
    Ghogiel greentooth

    you could just as well add an edit poly, select all the verts and connect.


    Though a turn to poly at the top shouldn't affect smoothing groups on editable polys in my experience.

  • Neox
    Offline / Send Message
    Neox veteran polycounter

    unless you have locked normals you shouldnt have issues with turn to poly. try adding an edit normals modifier prior to your turn to poly to see if you have edited/locked normals

  • Klunk
    Offline / Send Message
    Klunk ngon master

    what wrong with just right click and convert to mesh ? doesn't get any more triangular than that :?

  • Neox
    Offline / Send Message
    Neox veteran polycounter

    its completely destructive and also doesnt lock in the triangles, also not sure if fbx will do its own triangulation upon export or if it then takes the one from edit mesh

  • wandering_snail

    If I reset the normals I mess up my smooth groups again... I´m not sure if I did something with the normals in this model, I can´t remember to be honest but I think not.

  • Neox
    Offline / Send Message
    Neox veteran polycounter

    well there you have it, your normals are locked in, its not smoothing groups defining the shading.

    i'd suggest to reset normals and set up the smoothinggroups again. dependent on your needs it should just be 3 clicks in a script such as textools.

  • poopipe
    Offline / Send Message
    poopipe grand marshal polycounter

    Doesn't turn to mesh modifier respect explicit normals?

  • Klunk
    Offline / Send Message
    Klunk ngon master

    its completely destructive and also doesnt lock in the triangles, ?

    doesn't destroy smoothing groups or explicit normals and what does "doesnt lock in the triangles" even mean ?... the exporter will convert to mesh before exporting anyway all you are doing is looking at what's actually exported.

  • Klunk
    Offline / Send Message
    Klunk ngon master

    from the topoly.cpp

    // Luna task 747

    // We cannot support specified normals in Convert to Poly at this time.

    mm.ClearSpecifiedNormals();


    though tomesh.cpp


    // Indicate that we support the user-specified normals in this conversion.

    MeshNormalSpec *pNorm = m.GetSpecifiedNormals ();

    if (pNorm != NULL) pNorm->SetFlag (MESH_NORMAL_MODIFIER_SUPPORT);


    it's the same with editable_poly modifier will trash normals but an editable_mesh doesn't but strangely convertt to poly (on the right click) maintains them.

  • poopipe
    Offline / Send Message
    poopipe grand marshal polycounter

    Is that the same code used for the turn to mesh modifier?


    I think Neox is referring to the fact you don't explicitly force 3 sided faces by just converting to mesh from the right click - which can and does lead to edges being turned when imported into various apps and leaves you with busted normal maps.

  • wandering_snail

    Guys, I reset the normals and did the smoothing groups again and it worked! Thank you! I didn't understand half of the rest you said but yeah XD

  • Klunk
    Offline / Send Message
    Klunk ngon master

    yeah that's from the source for turn to poly and turn to mesh

    if your worried about the importer randomly turning edges once converted to mesh select all edges and set visible. But you'd have to do that anyway if that was happening (random edge turning) regardless as I said before the exporter is going to be doing the same (converting to trimesh) before exporting.

  • Neox
    Offline / Send Message
    Neox veteran polycounter

    The difference is, the turn to poly or turn to mesh modifiers do this in stack, non destructively.

    If you convert to edit mesh and turn everything visible. it is locked in, can not be undone, unless you keep a copy before that change somewhere.


    Also fbx export does no triangulate the same way internal triangulation works inside max. it totally does its own thing and isn't controllable the same way you can do in 3dsmax (compared to maya for instance where you have no controll over the internal triangulation of any ngon)

    if you export non triangulated to other apps, the output will be completely different to what max or the exporter does. so staying in sync will not work out, shading artifacts will appear.

    that being said, i didnt see 3dsmax based productions in a long while and usually we lock in the triangulation either in maya or the respective engine we use by flushing the meshes before baking through that.

  • Klunk
    Offline / Send Message
    Klunk ngon master

    Also fbx export does no triangulate the same way internal triangulation works inside max.


    fbx export doesn't do anything to the triangulation left to right original mesh, fbx export import with triangulate unchecked, fbx export import with triangulate checked. It produces the same result for both mesh and poly it also produces the same result if you randomly turn the edges so it's not just finding the same solution.

  • Neox
    Offline / Send Message
    Neox veteran polycounter

    My example stands, it's not like i am making meshes with errors on purpose :)

    What fbx export settings are you using? i can not reproduce this.

    As soon as i turn triangulation on it changes the way triangles are built from max. even with preserve edge orientiation. interestingly enough if i export an editable mesh, even if i turn on triangulation and keep orientation on export, it gets exported as quads.

    atm i use max 2020 to export, i tested a bunch of fbx formats, all result in the same.


    and another example

    fbx from max, quadded, returned from maya, internal edges are pretty much all flipped


  • Klunk
    Offline / Send Message
    Klunk ngon master

    what happens when you just export as fbx and then import back into max missing the maya step ? As this is what I get (max 2020)


    doesn't matter what settings you use the result is always the same

  • Neox
    Offline / Send Message
    Neox veteran polycounter

    the image with the spheres i showed is just that, exported an editable poly with triangulate on and reimported straigt back


    my very basic export settings


  • wandering_snail

    So, people, should I keep using turn to poly to triangulate or...?

  • poopipe
    Offline / Send Message
    poopipe grand marshal polycounter

    use turn to mesh instead

  • Ghogiel
    Offline / Send Message
    Ghogiel greentooth

    turn on Preserve Edge Orientation

  • Neox
    Offline / Send Message
    Neox veteran polycounter
  • thomasp
    Offline / Send Message
    thomasp hero character

    Perhaps that option exists as well in the FBX import module?

  • Ghogiel
    Offline / Send Message
    Ghogiel greentooth

    Ok yeah it's fubar

  • Neox
    Offline / Send Message
    Neox veteran polycounter

    but it gets hard triangulated on export? which would be okay as long as FBX is reliable and doesn't break anything. the upside of doing it via modifier is that it is based on the internal triangulation, which you can control in Max as opposed to Maya for instance.


    That being said, on import side you can only tell it to import smoothinggroups


    I'd still would like to understand what @Klunk is doing differently, as this is certainly something that could be more unified.


    i tried editable/edit mesh and turn to mesh modifiers, with those on. triangulation doesnt get applied at all on my end. but it keeps the internal triangulation in tact. i would bet if i now do the triangulation in maya or other softwares that would break


    i think i tried all possible fbx export combinations as well to have changed results


    but it doesnt matter, the only thing that is reliable on my end is locking the internal triangulation in, be it via modifier or by connecting all edges or turning them all visible in edit mesh.


    will download the latest zpdate and and see if it changes anything

  • Klunk
    Offline / Send Message
    Klunk ngon master

    I'd still would like to understand what @Klunk is doing differently, as this is certainly something that could be more unified.


    think it was a bug my end.... changing the settings was having no effect until I changed the version year :/ , after which the triangulate option worked on editable_poly (though still had no effect on trimesh), thinkl I'd just stick to editable mesh and or leave the triangulate unchecked. Don't think other software will change much once it's fbx

  • Neox
    Offline / Send Message
    Neox veteran polycounter

    oh it absolutely will if you do not lock in the triangulation. it's one of the most common error sources and the reason why many unreal projects flush their data through unreal into the baker. So they can stay with quads and work dcc agnostic to some extend.

  • thomasp
    Offline / Send Message
    thomasp hero character

    My bad, I did not see that checkmark for triangulate being set. The preserve option might be overriden with triangulation done some way in the exporter in this case.


    I'm regularly kicking meshes back and forth between Blender, Maya, Toolbag and an ancient version of Max. FBX or OBJ - my quads seem to stay intact regardless of where the geometry ends up in.

Sign In or Register to comment.