[Maya/SketchFab] ~ Folded quads, may I tell renderers what diagonal to favour?

Steamy_Steve

My problem: as you can see in the screenshots, the same quad in SketchFab and in Maya is presented with a different "interpretation".
[TOP - SketcFab]: the horizontal diagonal is kept intact and the vertical one folded; the result is an awful, steep drop in the profile of the rock
[BOTTOM - Maya viewport]: the same polygon, except this time the favoured diagonal is the vertical one, which perfectly follows the rock's profile as intended

My workflow:
.: Sphere3D in ZBrush, deformed and sculpted as hi-poly
:. ZRemesher in ZBrush to achieve a low polycount and then Project All to reapply hi-poly details
.: Exported low-poly as OBJ from ZBrush and imported it into Maya
:. Traced UV seams and re-exported as a new OBJ
.: Imported UV mapped OBJ into ZBrush to copy/paste UVs
:. Baked maps in ZBrush
.: Uploaded UV'd OBJ on SketchFab along with maps (just diffuse and normal)

As you can see, the uploaded OBJ is produced by Maya, thus I'd expect it to behave in the same way in both renderers. Which is not the case.

I can understand how, lacking any kind of explicit direction, different engines may render the same informations as they see fit.
So I've googled around trying to find a way to include such informations, but no luck.

What I'd like to know is whether it's possible to include a "fold this way" parameter in the OBJ file (or FBX, if it's better), or if there's any way to easily retopologize such tricky areas without creating awful triangles or an unacceptable amount of new quads.

For those who may want to ask:
.- yes, I'm an utter beginner
-. because I'm following a game design course and this was an assignment
.- no, our teacher haven't explained us how to effectively retopologize
-. yes, he's going to, but I just can't wait, this is how much my enthusiasm is pushing me
.- yes, you can criticize as much as you want, as long as you also provide advice
-. no, I don't do striptease....for free

m4dcow

You typically triangulate before baking maps, for weird areas like you have you could manually cut the face so it doesn't mess up the geometry.

I would also say that you should probably avoid baking maps in Zbrush, look into xNormal since it's free, but substance designer and marmoset toolbag have good bakers too.

Steamy_Steve

m4dcow said:

You typically triangulate before baking maps, for weird areas like you have you could manually cut the face so it doesn't mess up the geometry.

I would also say that you should probably avoid baking maps in Zbrush, look into xNormal since it's free, but substance designer and marmoset toolbag have good bakers too.

Why triangulating? =o
What if it's a model I need to animate? Would triangles still be good? =p

At the moment I'm more interested in a correct topology, rather than quality maps.
I need the model to work, once the canvas is ready I can paint whatever I want on it.
I already know XNormal (althought I don't have much experience with it), but the process would require exporting a ~500MB OBJ file, which takes time and is currently unneeded.

m4dcow

All renderers render triangles not quads (math thing with triangles) so ultimately quads get turned to triangles anyway. Which is why in the Maya viewport you notice the error with the quad being flipped the wrong way, it is because Maya turns the quads into triangles even though you don't see the edges.(there is a display option somewhere that can show the invisible edges)

If you really don't like what the geometry is doing, go into Maya and change it, don't have a quad splitting like that.

There isn't any real issue with animating triangles, skinning is based on verts so that wouldn't be affected. The only problem I would have is with editing (selecting loops etc) but you would save out a triangulated version, and have the quad mesh to go.back to if needed.

So the process of exporting high poly meshes is part of a typical workflow, if you think 500,000 is too much, decimate the mesh, but people generally export multi million polygon meshes without issue.

The reason I would advise against zbrush is that it's bakers aren't synced to anything, so they would likely look wrong in sketchfab and any other game engine for that matter.

Since you are just learning you can go and practice the workflows, but realize that there are certain technical issues you may run into.

Steamy_Steve

m4dcow said:

All renderers render triangles not quads (math thing with triangles) so ultimately quads get turned to triangles anyway. Which is why in the Maya viewport you notice the error with the quad being flipped the wrong way, it is because Maya turns the quads into triangles even though you don't see the edges.(there is a display option somewhere that can show the invisible edges)

[....]

So the process of exporting high poly meshes is part of a typical workflow, if you think 500,000 is too much, decimate the mesh, but people generally export multi million polygon meshes without issue.

The reason I would advise against zbrush is that it's bakers aren't synced to anything, so they would likely look wrong in sketchfab and any other game engine for that matter.

[....]

And is there any way to "turn" quads so that the invisible diagonal is in the right direction? =o
Anyway, so this means that Maya's triangles are always wrongly displayed?
I've found the option to make the extra edges visible, but I think they're turned the wrong way....?

I have no problems with exporting half a GB OBJ files, but I'm just testing some solutions, and maps/textures are definitely secondary, at the moment.
I could export the hi-poly version, then open in XNormal and bake normal maps that way....but it takes time and it's a waste of quality, for a test.

What means that "bakers aren't synced to anything"? =o

m4dcow

I don't think Maya has a way of spinning invisible triangles, and it's not that Maya wrongly displays them, its just that it might use a different algorithm.

As for bakers being synced, that is mainly to do with normal maps. Couple years back different game engines and programs calculated the tangent basis for their normal maps slightly differently, so the only real option was to do your best to minimize the differences that may occur, but you end up with normal map seams and artifacts. At some point game engines started to calculate their tangent basis based on a specific DCC app like Maya, Max etc... or a method of calculation like Mikktspace (which is the common one now). So when I say Zbrush isn't synced, I mean that it calculates normals it's own way and won't necessarily render correctly in another program.

rube

Maya uses the shortest edge for triangulation and it will change which edge it's using as you edit your model. If you need to guarantee an edge just split that face the way you want. It will also flip edges while objects are animating if the shortest edge happens to change.

Steamy_Steve

m4dcow said:

[....] Zbrush isn't synced, I mean that it calculates normals it's own way and won't necessarily render correctly in another program.

I guess that's why I have to uncheck the "Flip Green" option for normal maps in SketchFab....?

Anyway, to reduce polygons I use ZRemesher just because it tries to keep the mesh all-quad. You can always find a couple of triangles here and there, but the final result is remarkable nonetheless.
Decimation Master, on the other hand, doesn't give a cabbage about quads, and goes straight for an all-triangle solution.

Would you recommend a Decimation workflow, rather than a ZRemesher one?
Something like:
.: ZBrush sculpting
:. Decimation Master
.: Details projection
:. Low-poly OBJ export
.: UVs in Maya
:. UV copy/paste in ZBrush
.: Hi-poly OBJ export
:. Normal map in XNormal
.: Diffuse map.....Substance Painter, maybe?
.....?

Or maybe I could just take the ZRemesher result and simply cut the ambiguous quads in two triangles to prevent artifacts?

What makes me prefer ZRemsher is also the fact that I can use the "ZRemesher Guides" brush to set lines where I want edges to (preferably) run.
Like, for instance, all those nice crevices that are perfect to hide UV seams.

I definitely need a more solid, widely tested workflow, but the course I'm following has a one-lesson-per-week pace, and it feels sooooo slow to me. -.-'
It's good to have a whole week to practice the new notions, but every new lesson inevitably renders the previous informations partially wrong. It's like trying to correctly place a mosaic tile without having the whole picture.
Frustration level: 9999!

Steamy_Steve

rube said:

Maya uses the shortest edge for triangulation and it will change which edge it's using as you edit your model. If you need to guarantee an edge just split that face the way you want. It will also flip edges while objects are animating if the shortest edge happens to change.

This is quite a piece of information, I guess that explains certain artifacts that I did happen to witness in animations....

Steamy_Steve

@m4dcow
@rube

It seems I've found an effective method:
.: ZRemesher to reduce poligons while retaining selected edge loops (to favour UV seams)
:. Export low-poly as OBJ
.: Import OBJ into Maya, UV mapping + triangulation (you can triangulate AFTER UV mapping, seams are retained)
:. Export UV mapped/triangulated OBJ
.: Import UV'd OBJ in ZBrush to project details

And at this point I have a model ready for normal and diffuse maps baking.
I've already tested the traingulated model on SketchFab, and it's nearly perfect (minor things that are not related to my original issue).

Any further advice is warmly welcome, I'm open to try any possible alernative workflow.
Just remember I'm a total beginner, which means I can only use Maya and ZBrush, at the moment. If you intend to involve other softwares, please be detailed in your instructions. ^^'

m4dcow

Steamy_Steve said:

m4dcow said:

[....] Zbrush isn't synced, I mean that it calculates normals it's own way and won't necessarily render correctly in another program.

I guess that's why I have to uncheck the "Flip Green" option for normal maps in SketchFab....?

Not really, that has to do with "handedness" of the tangent space, but it isn't really necessary to get into at this point.

When you say project details do you mean projection as in creating texture maps?
You can also bake or project zbrush polypaint in xNormal (just remember to uncheck "Ignore per-vertex-color" in the high poly section).

If the object isn't going to deform and is an organic shape, decimating in zbrush does a really good job, it can just be a pain to cleanup and UV sometimes. If you are comfortable using remesher to get to other parts of the workflow you might as well stick with it for now.

You could also look at using quad draw in maya to create your low poly. You would bring in your high poly from zbrush, best to be a decimated one, make it live (in the modify menu). and then use quad draw in the modeling toolkit to make your geometry. This way you can have total control of your geometry and experiment with what works for you.

Steamy_Steve

[....]

When you say project details do you mean projection as in creating texture maps?
You can also bake or project zbrush polypaint in xNormal (just remember to uncheck "Ignore per-vertex-color" in the high poly section).

If the object isn't going to deform and is an organic shape, decimating in zbrush does a really good job, it can just be a pain to cleanup and UV sometimes. If you are comfortable using remesher to get to other parts of the workflow you might as well stick with it for now.

You could also look at using quad draw in maya to create your low poly. You would bring in your high poly from zbrush, best to be a decimated one, make it live (in the modify menu). and then use quad draw in the modeling toolkit to make your geometry. This way you can have total control of your geometry and experiment with what works for you.

No, when I say "project details" I mean I use the "Project All" function to bring the hi-poly details and polypaint of a sculpted model onto a decimated/remeshed model.
You can start with a subtool as hi-poly as you want, no need for a good base low-poly mesh; then, once done sculpting, you just duplicate the subtool, remesh/decimate it for a lightweight structure and then re-subdivide it while using the hi-poly model as some kind of morph target.
Veeeery useful, and works like a charm.

And now I've realized that you must already know about it.... *idiot* ^^'

Well, first I ZRemeshed it, then UV'd, and only AFTER mapping, I triangulated it.
This way I didn't have to struggle with the improbable shapes of a decimated model.
Basically I combined the usefulness of the ZRemsher Guides brush with the more visually solid results of Maya's triangulation funcion (which, by the way, DOES cut through quads in the right direction, and not just by picking the shortest diagonal!).

Odd that you're suggesting to use the Quad Draw tool, because just yesterday I stumbled on a video tutorial that was just about doing this exact thing: hi-poly in ZBrush, hi-poly export into Maya (GPU cache function be blessed!), Quad Draw + Live Surface, and back into ZBrush for details projection.
I did it and the resulting model wasn't even that bad, but I still had to realize where the problem lay and didn't triangulate, so I just kept getting the same visual artifacts. And thus discarded the method. ^^'
It also took me a very long time, another good reason that made me ditch it, but I guess it's just because the overall shape is pretty chaotic and I was trying too hard to include certain minor details in the low-poly mesh.

Now I'll probably try to furtherly reduce the low-poly polycount and then eventually start experimenting with Substance Painter, which our teacher mentioned as a tool we're gonna learn in this course.
I'm so excited that my guts are kind of quivering! x)