Normal maps and triangulation, baking in Maya
polycounter lvl 9
Hey polycount,
I've been trying to figure out the "proper" workflow for triangulating a mesh before baking and if it is even is a requirement.
From my understanding one of the reasons it is good to triangulate prebake is so that your normal maps dont get messed over when another app reads the normals differently.
However I have read that if I am making my models in Maya and baking it in Maya all I have to do is lock my normals before bake. But, then I also read that locking and unlocking normals is not reliable/ clients dont want it. Both of these come from two respected polycounters. Lastly I have not had many happy experiences with locking normals, though that may be due to user error. So my question is should I be triangulating in Maya before bake?
Im a bit weary of doing it though because it becomes a pain to edit after the model has been triangulated. I know I should probably make a duplicate then traingulate but it feels like an necessary evil?
I will be learning how to bake normal maps in Xnormal just to be flexible and from what I gather, triangulating is neccessary. So I guess Im wondering what other peoples workflows are when triangulating and baking.
And finally a lot of marmoset viewer models are not triangulated, so am I missing something or is everyone just skipping that part as well. But then Im sure alot of people use Xnormal so what do ._.?
cheers!
I've been trying to figure out the "proper" workflow for triangulating a mesh before baking and if it is even is a requirement.
From my understanding one of the reasons it is good to triangulate prebake is so that your normal maps dont get messed over when another app reads the normals differently.
However I have read that if I am making my models in Maya and baking it in Maya all I have to do is lock my normals before bake. But, then I also read that locking and unlocking normals is not reliable/ clients dont want it. Both of these come from two respected polycounters. Lastly I have not had many happy experiences with locking normals, though that may be due to user error. So my question is should I be triangulating in Maya before bake?
Im a bit weary of doing it though because it becomes a pain to edit after the model has been triangulated. I know I should probably make a duplicate then traingulate but it feels like an necessary evil?
I will be learning how to bake normal maps in Xnormal just to be flexible and from what I gather, triangulating is neccessary. So I guess Im wondering what other peoples workflows are when triangulating and baking.
And finally a lot of marmoset viewer models are not triangulated, so am I missing something or is everyone just skipping that part as well. But then Im sure alot of people use Xnormal so what do ._.?
cheers!
Replies
You triangulate before baking not "so your normal maps dont get messed over when another app reads the normals different" (as you say it — the normal map is being read complete fine/correctly), you triangulate so that the baker loads the mesh with the same triangulation as it will be displayed with, as triangulation affects surface shading (which therefor affects your resulting tangent-space normal map)
In xNormal you potentially could avoid triangulating before export iff (if and only if) you can get a triangulator plugin that will match your software's triangulation (maya in your case) so you won't have to worry about them being different. But it's usually just simpler to triangulate before you export, and that's more universal anyway.
I guess its a bit petty, I dont really like the idea of having to duplicate because it leaves my scene messy. However its thats what has to be done i'll bite.
What really threw me off was seeing marmoset viewer models that were quads. Also in normal wireframe shots a lot of models are still in quads, am I missing something really simple here?
If you simply triangulate in Maya before baking, you can skip the lock normals step, but then you have to make a copy of your mesh, or undo the triangulation after baking. For the most part I've managed to bake in Maya without triangulating first, but I've had a few rare occasions dealing with meshes with mirrored uvs that had different triangulation direction for the quads on one side which required manual triangulation.
The reason it is important to triangulate before baking or exporting to a different game engine/app is that the way quads are converted to triangles internally in apps can vary, which will lead to X shaped smoothing errors.
Are you missing something? Some artists simply do not understand the important of triangulating meshes for baking/export. For certain types of assets, the artifacts that triangulation will prevent will be less noticeable. For instance, with organic shapes or character art where you don't tend to have a lot of harsh angles, smoothing errors in general tend to be less noticeable. For hard surface, mechanical, armor, etc, where you have more harsh shapes and often have very reflective metal surfaces, smoothing errors can be very apparent.
However I do have a problem with locking normals. I set my edges hard/soft but when I lock the normals it hardens all the edges adjacent to any existing hard edge. Hard edges are on uv splits.
And I'm sorry if it sounds like i'm going crazy over this, it's not that drastic, I'm just curious is all.
This was an example I was wondering about, I'm sure Amsterdam Hilton Hotel knows what he is doing and I know you know what you are doing. Its a hardsurface asset that hasnt been triangulated. The gun looks amazing but is it considered bad practice?
Image taken from http://www.polycount.com/forum/showthread.php?t=158840
So how are people posing their characters with normal maps applied and their wire frames untriangulated?
They are either not triangulating, comp wizards, or im an idiot.
Ben uses Max, in Max, your mesh is always triangulated even when working in quads, the invisible edges are stored and can even be edited if you like. Unlike maya, changing from quads to tris has no effect, so you can safely bake with quads in Max, and export as triangles as the last step. Alternatively, you can export as triangles and bake in an app like xNormal.
As far as character art goes, again, character art content is frequently much more forgiving to these artifacts, so it's possible the problems are simply not noticeable on the work you've seen, thus not really worth worrying about. As with everything, you should do some real world tests to see if it makes a difference in your workflow, with the assets you work on.