"Realtime" mesh displacement mapped and animated

S-S

- Second take on displament mapped characters from yesterday and today (posted similar stuff in zbrush forum last year)

- I have been trying to develop workflow for myself for diplacement mapped characters - it is actually the same procedure as making normal maps in zbrush and then using them in 3dsmax or games... but haven't had enough time to do tests
- I wonder if nextgen consoles like xbox2 and ps3 will have displacement map support for higher detail level... it could be stored and used in addition / replacement for normal maps

- Yesterday I took my old test mesh, fixed loops, and made changes using tools in the way, that i wouldn't corrupt UVs actually when i was back in zbrush, i had lost about 75% details, i had to sculpt the whole mesh from scratch - I changed proportions and head style - i also had to re-do hand and foot too

- Today went back into max, re-skinned it to old skeleton - Here you can see meshes in zbrush:



- Here is the animation, based on tweaked mocap clip (quicktime/mpeg4 1.3mb) - Click here if you want to see the test animation:



- Here are some frames from the animation:



CnC welcome!

P.S.
I'm currently looking for any modeling freelance jobs, low or hipoly or anything which has something to do with 3d graphics!

Liquidmetal

Is the clip of the low poly model with the displacement map?That looks really cool.How dod u make the displacement map?I really like the low poly model,because I am studying edgelooping atm.Gj!

S-S

Liquidmetal:

Yes, animated mesh is the "cage" which you see in the right side of zbrush screencap - This in turn is just displacement mapped in 3dsmax (similar process to using normalmaps in 3dsmax)

Wrath

Maybe you can clear a few things up for me, I think it semantics...

I've always considered displacement maps to be a texture (generally a heightmap) that tesselates and offsets the lower polygon geometry at rendertime. This creates very high poly geometry to be rendered. The benifits are that you can more easily texture fine details than model it, and as I understand it this is basically how ZBrush works.

Normal maps, on the other hand, fake high poly geometry or at least the way that high poly geometry would light by calculating the angle of incedence on each pixel of the normal map. This is kinda the best of both worlds, very few matrix transformations for the verts, but fairly high poly appeareance from the lighting. If the normal map is very low resolution, then you wouldn't get very good results because at some point the pixel size of the texture would be larger than the tesselation of the geometry.

So the benifits of displacement over normal map would get the sillhoute of the geometry...you wouldn't be able to see the low poly edges like you could often see in Doom3. Now I know there are some other pixel shader tricks you can do to get faked displacement by playing around with the Z-Biasing (Unreal3 does this) but I don't think it affects the contour. Without specific hardware acceleration for this particular feature, I can't see how it would be more effecient thatn just throwing high poly geoemtry to the card, but we are getting into something beyond my experience.

So I guess what I'm asking is, what exactly are you showing off here?

S-S

Wrath:

"what exactly are you showing off here?"

- Well, i think you know better than me how much pain in the ass skinning can be (checked your site) - i like the example of tube, which show what the transformations created by bones actually are - just subdivide it enough and you wont be able bend it believably - no matter what. - Next solution is to bring in additional bones, and morphs to correct the shape. Too many bones is a problem for deformation/animation and ingame use IMO.
- You can see in games like Dead or Alive 3, Tekken Tag Tournament, Ninja Gaiden etc. how the high res meshes act on extreme angles (well, i think they have "faked" some press images) like ninja gaiden, when main character raises his arm up and front, it looks more like woody from toy story . In DOA they avoided this by removing polys from underside of shoulder to overcome the problem of very high poly densities (i don't think morphing is an option in xbox with 25000 poly cages)

- What i was trying to say; - i kinda just thought about gaming uses of displacements, but NOT THAT seriously now displacement maps are used in movies like LOTR, and probably will end up in games some year sooner or later - maybe.

- And what i was after with this test - it is easy to setup the mesh which is low res/medium res - it is easy to animate and additional shapes are stored in map, which is added on top of the animation cage, removing the need to model, or get headache from weighting/rigging it. - i don't think muscle simulations will be available in nextgen games - it is really hard to do with current homecomputers even for rendering purposes - well, only talking on behalf of myself

- Sorry to cause confusion!

P.S.
"Z-Biasing" - is this similar to silhouette mapping or profile mapping stuff in sigraph papers? - I remember reading about some kind of tech to store profiles of object from different angles to fake profile of low res mesh? (i think it was only for non-deforming objects...)

MoP

I think Z-Biasing is the effect used in "parallax-mapping" - slightly different method of displaying a normalmap, where it actually shows pixels as being "in front of" other pixels, depending on the angle - seen in the brickwork in the Unreal3 demo videos, where they look really 3-dimensional (more so than just regular normal-mapping), while still being on a flat plane. It does not affect the silhouette of the mesh, as silhouette-clipping would.

Anyway, yeah, I like this. It's the mo-cap that makes the animation look cool, but I understand why you did this test, and the overall effect is very good. Although I think it will be a while before we see this getting used in games, at least another year or more. Probably much more relevent to film.
Why didn't you go and use textures and sub-surface-scattering too, and make this look really realistic? That would be a cool comparison - show the low-poly mesh with the same animation, and a simple diffuse texture map, then compare it with the displacement-mapped and detailed mesh with advanced lighting.

Good work.

MoP

S-S

MoP:
Thanks

"Why didn't you go and use textures and sub-surface-scattering too..."

- Yeah sure Maybe i will refine it later - i also need better renderer and real displacement (i only used high subdivision setting in meshsmooth - level 5 will crash my max (out of memory)

Wrath

I definitely agree with you about deforming geometry being easier without having to deal with a tremendous amount of tesselation, I guess what threw me off was the "realtime" in your topic. From what I understand, you're just doing displacement for the rendering, which isn't realtime. So this is like a proof of concept for what realtime displacement maps might do for you.

MoP, I've worked a bit with "parralax mapping" in the past and it's completely seperate from normal maps. It's actually more of a 3 dimensional effect than normal maps because it simulates the parralax effect on geometry rather than just simulating lighting of higher geometric detail, but it's very subtle effect on it's own. When you combine it with normal maps it really pops.

I used to think we were going to move toward higher geometry, but I think we're just going to move toward more tricks to make mid poly geometry appear to be high poly. Normal maps, parralax maps, & 'silhouette-clipping' can effectively give you the same appearence and render a lot quicker than displacement maps and tesselated geometry.

Whatever it ends up being, on the content side we'll still be modeling (or painting displacement maps) for super high polygon models and piping them through some sort of processing to fake it quickly in the games. I'll just be glad when the tools can do it all flawlessly and automatically.

Regardless, it's a nice model (and displacement map) and with the mocap appears very realistic

Daz

Very interesting and well done S-S. Btw, I am absolutely certain that PS3 and Xbox2 will be capable of displacement map technology.

CheeseOnToast

I'm not holding my breath. "Emotion Engine" anyone? *kof,kof*
Great work there S-S. You had much luck generating normal/disp maps in Z-brush that don't have little artifacts all over them? If so, I'd like to know the secret

S-S

Daz:
- i was thinking about those nextgen "Madden 2007" images which have been shown on certain small gaming sites lately, really nice higher dynymic range lighting, can't say about much about mesh resolution or if they are "real" ingame images or just clever marketing trick

CheeseOnToast:

- I have used adaptive setting for displacements, adaptive has worked better than no adaptive IMO (just render many different displacement maps with different settings and you'll see how zbrush makes them!)
- Another factor is that i have only used mesh subdivision, not subpixel/adaptive tesselation tech used in other renderers like Vray, fR, or Mentalray - this too might remove some artifacts, as resolution isn't infinitely high
P.S.
I don't think it is very good idea to do normal maps in zbrush - zbrush can't currently handle overlapping geometry that well - it renders these areas incorrectly

Kevin Albers

I don't think standard renderers have support for parallax displacement yet, so doing renders using tessesating displacement is in fact probably the best way to test out displacement maps that would be used in parallax displacement for games. It would be nice if something like Max supported rendering parallax displacement, in order to preview objects before importing into a game. I think parallax displacement maps in games will become common pretty soon, since they give such nice bang-for-the-buck in terms of quality vs render time.

EarthQuake

haha

gauss

very nice work S-S, please continue to do more and post your experiments that little animation test reminded me of the snippets posted up on www.amazonsoul.com ... they've got some really nice looking characters on that site. keep it up.