Normal Map Projection Problems

Chiffy

Hey guys, so this issue has stumped me for a little bit now. Basically I've made a cage for my high poly model, seen here (the pink is the high poly showing through the low poly, however the cage is still enclosing both):


and the problem is that even though the cage appears to completely enclose the high poly, I'm still getting results like this (appears like there's parts of the model the cage isn't capturing:



Is it possible I'm getting these errors because the high poly is too different from the low poly? Or should I crease the low poly's edges? My process for making the high poly has been to export the low poly using GoZ to Zbrush, then reexport it back to max, add 2 Turbosmooth modifiers (checking use smoothing groups on both), and reexporting back to Zbrush for further subdivision. Doing it this way allows me to keep the shape pretty well without using crease or support cuts. When I've tried just exporting the low poly to Zbrush and subdividing it with creasing in there, I get lots of faceted faces, that aren't smooth at all and require me to go in there using the smooth brush to smooth out the model so I don't get striations on my normal map like this:



If anyone has a better way of taking a low poly you're happy with and subdividing it feel free to let me know, as I'm new to all of this. Just seems like normal maps are giving me more of a headache than they should. Thanks again guys!

Chiffy

Anyone have any thoughts?

Bartalon

Looks like you have some smoothing group issues. Set each UV island of your low poly to its own smoothing group and try to re-bake.

You might want to split the UVs of those crystals also.

[edit]

Oops, I totally missed that paragraph between your images!

It sounds like you're giving yourself a lot of extra work with importing and exporting and whatnot.

If you're starting with a base mesh in Max, just send it over to ZBrush and start sculpting. If the first couple of subdivisions causes your mesh to lose too much volume, turn off Smt next to the Divide button, then turn it back on after a couple divisions. You shouldn't have to keep bouncing back and forth between software so much.

As you sculpt, your lowest division will shift slightly, which is to be expected. If you're not going to retopologize, then the lowest subdivision's resulting shape after sculpting will be what most closely matches the details you have added in your higher divisions.

Either way, you should export your high and low and bake out your normals in xNormal. ZBrush tends to produce questionable bakes in my experience.

Chiffy

Thanks so much for the advice! This really helps, I tried out just turning off smt, for the first 2 subdivisions, then turned it on for the rest. (Also realized I was using the low poly I imported to Zbrush, instead of the lowest sub level low poly! Haha!) It gave me pretty much the same results as creasing the whole model does, which is cool. Problem though is that a lot of areas of the high poly still need manual smoothing with the smooth brush, like the poly's are faceted. Turning off smt for the lower sub levels prevented these faceted polys from being smoothed, but more of the original shape was retained. I guess there's no way to improve this other than specific controlled creasing? I am using xNormal for the bakes by the way

Key question I have however, when do I need to retopologize my high poly? I imagine if I already have the major forms in place and am just adding details like designs on the armor, and scratches, then I don't have too? Thanks again, really appreciate it!

Bartalon

I think a more modern approach is to begin in ZBrush without a base mesh, then when your high poly has been set in stone, you retopologize it to get your low poly. From there you can unwrap and make your cage, then bake everything in xNormal. This method minimizes how much you have to swap between software.

Starting with a base mesh can be a very rigid approach to character modeling. If you need to add or subtract major forms, you have to hop back into your modeling software to do that which is just a big interruption to your work flow (not to mention all the facet smoothing you talked about). Starting high and not having to worry about how your base mesh will deform (because there isn't one yet) is definitely liberating.

If you are using the model you have as a basic blockout for your high poly, then you shouldn't worry about a cage or any of that till your high poly is completely done. I would recommend looking into creasing to make your base work in your favor for a high poly blockout.

Retopologizing happens after your high poly is done. Sometimes you can retopologize your high poly pieces to clean up the edge flow, such as if you have a rough dynamesh sculpt and need to re-route the edge flow for creasing/hard surface details.

Chiffy

Wow you're right, starting high does seem like it makes a lot of sense...lightbulbs are clicking on in my head haha. Yeah a base mesh definitely seems to complicate things...especially with the deformation of the model being altered after it's been sculpted on as you said. For this model I started low poly though, but I know what to do for the future haha. I'll look into creasing for the time being. I also found Baj Singh's method here on PC for low to high poly worked a lot better than what I had before, here's the new normal map I generated (still subject to some improvement, pending creasing and additional smoothing etc..):



Thanks again for all the help and advice. I see how retopology fits into the whole workflow now much better (btw UV layout is unfinished, this is just a test)