Uncharted 4 Cloth deformation technique?

melviso

I have been following Naughty dog's work for sometime now and I am beginning to feel that they use a lot of blendshapes or morphing technique in order to nail down realism.

https://www.youtube.com/watch?v=UmD2MVlPGGk

This is the latest video from ND and I was actually taken back by the characters taking off their suits. This is something that is generally avoided by 3d animators. I am guessing they had to keyframe the vertex animation or use blendshapes for every frame. I also noticed they are used this technique for Nadine's hair. Due to the huge data, I am guessing they can't use it for every scene.

I must say ND are taking the pains to add really subtle details and that is really commendable of them. The scenes I have seen so far are great . The only giveaway are how plastered the clothes are to the characters.

thomasp

might be one-off baked simulation, like the type crytek used on ryse (info is in some GDC or siggraph paper). if so, they surely don't use it for the hair (which looks great, btw). the hair animation looked like realtime physics (or physics baked to bone chains) to me anyway.

you could probably whip something up with a mixture of blendshapes and physics, too but any revisions on that and the animator will run home at lunch to fetch the shotgun.

i'm surprised that they do all this and then choose to not use blended normalmaps for the folds? should work a treat on suits and dress shirts. or it's there and i missed it. there's something strange going on on my machine with that video anyway. has a grey overlay to it, browser-issue or some intentional flashback-effect?

Scruples

It's definitely baked as it bounces somewhat unrealistically when the first guy takes his suit off. We will probably just have to wait for the inevitable white paper detailing their techniques. I'm more interested in the hair, looks great!

PolyHertz

Yea, I'd say it's baked.

melviso said:

I have been following Naughty dog's work for sometime now and I am beginning to feel that they use a lot of blendshapes or morphing technique in order to nail down realism.

They do. NDs character rigs use a combination of joints + corrective blend shapes + wrinkle maps. They're doing some new collision based deformation with UC4 too that they've shown off a bit in some other videos.

Mark Dygert

They are very cleaver with the camera positioning and cuts, just about the time or place it would fall apart it goes off screen or the camera cuts away. You can't blame them, cloth for movies is a tricky enough beast, let alone trying to dump it down for real time.

joebount

Bump with a related question: what would be the best way to author the wrinkle maps? I heard them saying they were using scan data.

How would they author them? take a positioned scan (flex, twists, other...) then match their base model and bake (if so, how can they match precisely the scans?). Would they instead take the bits they want to pop up and project them on the base model. Would they use them as simple refs ?

I would be interested into hearing what you think and also if you have any reading material about the subject

PS : anything related to blend shapes would be great as well.

Cheers

FourtyNights

It's best to have the lowest subdivision level in ZBrush being the final low poly. That way you'll get both high poly and low poly sharing the same exact positions of expressions for wrinkle map bakings.

For example, my workflow is to sculpt the character's head with whatever topology is good for sculpting, all the way to the final secondary forms (I don't care how messy the base mesh topo gets during sculpting, as long as the sculpt's anatomy is looking top notch itself). Once it's done, I'll manually retopo it in Blender with a quite dense AAA game rez animation-friendly final topology. Then I'll bring the new topo back to ZBrush on top of the original sculpt, subdivide it as high as the original sculpt's density is, and use ZBrush's fantastic "Project All" tool sub-d by sub-d starting from lowest to the highest.

After all that I'll sculpt the final tertiary details and facial expressions with wrinkles to 3D layers.

joebount

hum, it's a bit more tricky than that. I am talking more about the alignment of scans in that precise base position.

The issue is when you are trying to blend a wrinkle map above some details, you need these details to stay precisely in place of you will have some floating issues.

If it's just sculpting this is not a problem, you can work with layers, share the same topology, etc.

Thank you anyway, always nice to read about workflows

thomasp

i imagine scans are taken mainly for reference. perhaps they use a tool (like we did back in the day) to adopt the main facial shapes as a starting point to build expressions. 

their work still looks somewhat painterly, not like dead-on photogrammetry, at least from what i've seen. it doesn't look like it would require totally sophisticated workflows uncommon elsewhere just done with experience, more care and probably more time invested than what others usually are willing to afford.

joebount

thomasp said:

i imagine scans are taken mainly for reference. perhaps they use a tool (like we did back in the day) to adopt the main facial shapes as a starting point to build expressions. 

their work still looks somewhat painterly, not like dead-on photogrammetry, at least from what i've seen. it doesn't look like it would require totally sophisticated workflows uncommon elsewhere just done with experience, more care and probably more time invested than what others usually are willing to afford.

Thanks dude. Yeah, that's my guess as well. I was wondering if they might have some kind of secret sauce, since that aspect hasn't been very documented (that and the fact that it's one of the few places I know where they are actually using scans)