Retopologizing garments and baking overhanging details

MagicMeister

Hi all,

I'm new to the forum but I've read a lot of wonderful info on here and this community is amazing so I figured I'd give it a shot to ask here before I try and reinvent the wheel on my character project.

Concept:

I have a few questions regarding the retopologizing into baking normal map work flow for game characters.

What you can see on my character model are a lot of thin, overhanging flaps on the sleeves of her shirt. I'm wondering if anyone has suggestions on the best way to retopologize something like this so that the overhanging VDM details are actually transferred into the final low poly via normal maps.

The reason why I'm anxious about this whole process is that I've remade this shirt a few times already, each time with a new method, because the geometry simply refuses to project / zremesh from dynamesh into a clean topology. This time I've been more cautious, duplicated older versions, and applied the leather flaps on their own layer as opposed to a destructive workflow, however now that I have a clean, subdvided high poly, I'm totally worried about the retoplogizing and baking phase. Could I retopologize this with flat faces following the form of the sleeve like a cylinder, or do I need to manually retopologize every little fold, divet and overhang in order to make sure that the details actually bake proeperly?

Thank you so much for any insight. If I'm just approaching this completely wrong I'm open to new workflows or suggestions.

Also, when retopologizing clothing, do I need my low poly to account for the thickness of the high poly material, or can that also be baked in via normal maps? Like can it just be the edgeloops and vertices without any thickness, or do I need to include vertices and edgeloops for the width and inside of the garment?

dimwalker

imho, Just bake HP to cylindrical LP arm - easy to manage and tweak, clean loops. If you want to pump it up, go for parallax occlusion or even displacement. Making each individual scale would be a pain to model, skin and animate while you probably won't even see it in game, apart from close-ups in cinematics.

Personally I would stay away from real thickness of garment. It would probably intersect itself a lot during animation/simulation.

MagicMeister

Thanks for the suggestions! I was just worried that the baking process might resist a lot of the overhanging sections, but knowing that more experienced modellers would still approach this with a cylindrical LP arm is encouraging.

I'm expecting to do some back and forth with tweaking the bakes once I get there but as long as I'm headed in the right direction with it then I can feel better about moving on to other parts of the HP sculpt.

Klunk

looks like you could reproduce that effect in lower res as a set of overlapping  bands with an alpha then space them down the arm, running a gradient from top to bottom of each band would add additional depth. create a few linear  variations of the feather theme to render to texture then just map the bands to those.

MagicMeister

Interesting, I like that approach as well. That would certainly maintain the depth and the feeling of layered, leather tiles. Once I start the retopo I'll come back here and update this thread with the various approaches and share what works.

pxgeek

Regarding overhanging details: normal maps won't really capture that. (Think normal mapped tiled roof texture as a similar comparison)

You could always do a quick test bake (for the simple cylindrical topology method) just to see if it looks good enough for what you want.

fwiw in the case of Diablo 4 models it's most likely what they're doing.

MagicMeister

Yeah, that makes sense for D4, but I was thinking of having this be more like a Baldur's Gate 3 (i know that's a tall order for a beginner, but I'm just going for it and I'll see where I land). I'm not going for cinematic level quality, but it would be nice to keep some ambient occlusion underneath the larger flaps.

Klunk

I did a quick test and it seems to work after a fashion, the shaded gradient in each ring is a simple vertex shading
and it use alpharef (0 or 1) for the alpha.


obviously needs more work  
some of the issues are there's quite a lot of over draw and at acute angles you can see through the mesh if each ring differs to greatly from the next (though too close may cause clipping issues) so may require addition internal solid geometry to hide this adding to the overdraw

MagicMeister

The gaps between the rings could definitely be covered up with an internal piece of geometry which also makes sense conceptually as these little scales or flaps of leather would be attached to a sleeve. I like that solution, but I'm going to test out a retopo cage that accounts for some of the overhangs and then try and push the detail with vector displacement. I may not get to it until this weekend but I'll post the outcome when I give it a try.