How to merge a Sculpt with Rocksurface the right way

reiro · Mar 2010

I ran into a problem where i need to merge a seperate head sculpt into a rocksurface in Zbrush without making a big mess. The whole meshes will be part of an Level/Map using the Unreal3 Engine. I got several problems i ran into along the way.

1. PROBLEM - losing detail

I sculpted a Rocksurface for my level seperately which will make up the cave where the level sits in. It is about 1.6Mio polys in Zbrush.

However my Headsculpt is about 1.6Mio Polys alone, so i am losing a lot of the heads detail when trying to project the detail onto the rock surface
.
Even if i up the max amount of polys of the rock to 6Mio it still not looking right.

2. PROBLEM - messy topology

Since the head is sculpted from a plane and isnt retopologized yet and the rocksurface being uneven and not flat it gets very messy when i try to project the head onto the rock. I tried various ways in Zbrush, like projecting with subtools or using projection master. The results where extremely messy and hard if not impossible to cleanup.

QUESTIONS:

What is the best solution to have a game friendly topology later on? Retopo the head in zbrush and make it separate to the mesh? Retopo the whole rock with the head merged?
Is it better to keep the rock and head seperate meshes and just stick them into each other in the UDK? Or is this not a good looking and clean solution?
I really dont know in what direction to head atm so i would be glad for any suggestions or ways how you guys would go about this "problem"

To get more of an idea i attached a pic that shows the rock/head which is seperate atm.

Eric Chadwick · Mar 2010

Hard to get an idea exactly what you want to do. Do you have any concepts of the intended result?

To make the head look like it's carved out of the rock wall, I would make it using two maps. Position the head where you want it, retopo the wall including a separate-able section for the head.

Then I'd bake the rockwall using the complete retopo, and bake just the head using the head section of the retopo mesh.

In photoshop I'd open the head section, put the rockwall in a layer below, and erase the edges of the head. Make sense?

You'll still have uneven resolution between the rockwall and the head, but you'll have a better blend. You could also use a vertex blend instead.

reiro · Mar 2010

EricChadwick wrote: »

Hard to get an idea exactly what you want to do. Do you have any concepts of the intended result?

To make the head look like it's carved out of the rock wall, I would make it using two maps. Position the head where you want it, retopo the wall including a separate-able section for the head.

Then I'd bake the rockwall using the complete retopo, and bake just the head using the head section of the retopo mesh.

In photoshop I'd open the head section, put the rockwall in a layer below, and erase the edges of the head. Make sense?

You'll still have uneven resolution between the rockwall and the head, but you'll have a better blend. You could also use a vertex blend instead.

To get this straight. You would make 2 different meshes. One for the whole rock and one with the head plus a bit of wall which will be both baked separetly?

"...Position the head where you want it, retopo the wall including a separate-able section for the head."

With position you mean not to merge it with the rock in zbrush but to treat it as seperate meshes? Or you mean doing the whole wall plus the head which will be higher in poly count which will be merged with the lower walls?

As for the rest of the rock i initially planned on just decimating the whole rock as one mesh. Then cut the low decimated mesh in 3dsmax into managable chunks, uvmap it, bake it and puzzle it back together in the udk.

divi · Mar 2010

since you don't want to use tiling textures for the rocks i'd "just" retopo over head and rocks, project both onto the new mesh and clean up. or leave the meshes seperate, bake them on the low and clean up the seam between face and rock in photoshop.

Justin Meisse · Mar 2010

personally I'd just do a tiling rock texture for the ceiling, you could build the head into the rock mesh and use vertex blending to transition from the tiling rock to the uniquely wrapped head.

Jesse Moody · Mar 2010

Justin_Meisse wrote: »

personally I'd just do a tiling rock texture for the ceiling, you could build the head into the rock mesh and use vertex blending to transition from the tiling rock to the uniquely wrapped head.

Yep probably the same way I would go on something like this as well.

reiro · Mar 2010

Justin_Meisse wrote: »

personally I'd just do a tiling rock texture for the ceiling, you could build the head into the rock mesh and use vertex blending to transition from the tiling rock to the uniquely wrapped head.

Yea that would ve worked if i havent uniquely sculpted every rockwall.
It will take some more time than if i just had used a tileable texture but probably adds to the looks of it. If i would do it again i probably would ve gone with tilable rock texture but too late now. Dont want the hours put into the rock sculpt put to waste.

What about adding a bit to the head to merge it more with the other rock and then retopo it seperately and just decimate the rest of the walls?

Like so?

I ll keep on trying. Thanks for the different suggestions.

Justin Meisse · Mar 2010

I don't think the sculpted rock is a waste if you use a tiling texture, you can retopo what you sculpted for the low poly shape and apply the tiling rock to it. If you're just doing this for fun, go ahead and do whatever you want but if it's a portfolio piece I'd be concerned.

If you're dead set on not using a tiling texture you can still build the low poly encorporating the face into the wall, lay out the rocks on UV channel 1 and the face on channel 2 bake out the two textures and then do a vertex blend.

Eric Chadwick · Mar 2010

Also I'd recommend limiting the head UV to just the head and a little bit of wall, not all that larger yellow area, to maximize the size of the head in your 2nd texture.

reiro · Mar 2010

Justin_Meisse wrote: »

I don't think the sculpted rock is a waste if you use a tiling texture, you can retopo what you sculpted for the low poly shape and apply the tiling rock to it. If you're just doing this for fun, go ahead and do whatever you want but if it's a portfolio piece I'd be concerned.

If you're dead set on not using a tiling texture you can still build the low poly encorporating the face into the wall, lay out the rocks on UV channel 1 and the face on channel 2 bake out the two textures and then do a vertex blend.

I see what you mean. So you mean using the uniquely sculpted rock break it into pieces, uv it...
and then use a tilable diffuse/normal/spec for the rock? So just screw the uniqueness in order to have something that would be actually done in a industry workflow? Do i get you right?
Iam aware that in a production pipeline you probably wouldnt do this due to time and performance constraints. I just thought to push it a bit further. But you are probably right. Its still a game mesh not something for a rendered movie.

So basically i just need to sculpt a tilable rock segment which i ll then use for the entire piece? Did i get you right? Basically like this? http://vimeo.com/7510490

And then probably doing 2 different variations for vertex blend? Like one with moss one without?
I have to note that there are 2 holes in the ceiling where light will shine in, so there will probably more "green". So i guess i ll vertex paint it then to save texture space.

Eric Chadwick · Mar 2010

Links to more tilable-rock tuts (including that one)...
http://wiki.polycount.net/CategoryEnvironment#ES

hijak · Mar 2010

i dont see why not use 2 intersecting meshes, one for the head one for the rocks. The seam will be visible at first, but once you do an ao pass you wont really notice it. It will most likely seem like it was a natural seam. Even if it was carved straight from the rock i imagine there would still be some sort of visible transition.

if your concerned with loosing your sculpted rocks but still want to be albe to use some tileable textures then try this.

create normal maps from the sculpted rocks as ussual, prob ao maps too just in case you need them.
Have this nrm map be the main one and responsible for the large details.
now have 3 tiling maps diff spec nrm. This will be tiled a few times to give smaller details.

so your basically just doing tileable diff and spec with a composite nrm map. One is the large scale one is the small scale to help add detail while retaining a small budget.
then you can even have the ao you baked earlier multiplied over the whole thing in the shader, for that extra realism.

for this type of organic terrain i think utilizing tiling textures multiplied over unique details is really important, and also a huge benefit to your budget.

my explanation is not the best but people do this all the time in unreal, and it was pretty easy to do it in maya via cgfx shaders as well.
and i should have mentioned that ways i would try are using 2 uv sets one for the large scale nrm and ao and a second set for the tiliing stuff, somethings this can be unitized or automapped or whatever, depends on your topology.
the second way is to make a shader that allows for the tiling of the detail set to be controlled separately from the tiling of the large scale stuff. That would only require a single uv set.

reiro · Mar 2010

hijak wrote: »

my explanation is not the best but people do this all the time in unreal, and it was pretty easy to do it in maya via cgfx shaders as well..

create normal maps from the sculpted rocks as ussual, prob ao maps too just in case you need them.
Have this nrm map be the main one and responsible for the large details.
now have 3 tiling maps diff spec nrm. This will be tiled a few times to give smaller details.

Kinda makes sense but wouldnt that be a lot of different maps? I mean the cave if uved is probably more than one UV Map since its quite big. If you then have a large scale normal and a detail tilable normal isnt that exeeding the budget?

Iam only concerned when you say "this is how people do it all the time" that its not the way it is good for your portfolio. At the end of the day i wanna have a good portfolio, this isnt only done for fun so iam a bit concerned whats "okayish" in industry terms.

I assume every way has its downsides. If you make it tilable you are losing out on lots of unique detail, if you make things too unique you exceed the budget and its not really made for games anymore.

Is the last method "ok" in terms of portfolio work? Or is it better to simply do one tilable texture and work that uniqueness back in with vertex paint? To show the rest of the cave to have a further judgement...i again attached a pic.

If i make it entirely tilable i lose the detail like the cracks...what are you normally do in cases like that? Different Normals? One rough one tilable? Or some sort of "crack maps"? which will be vertex painted?

Lots of different methods and it got even more confusing atm

reiro · Mar 2010

EricChadwick wrote: »

Also I'd recommend limiting the head UV to just the head and a little bit of wall, not all that larger yellow area, to maximize the size of the head in your 2nd texture.

True indeed.

Justin Meisse · Mar 2010

the problem with the unique unwrap is how big will the texture be? Chances are it will still have a lower pixel density than the rest of the scene.

I recently had to rework some assets that were uniquely unwrapped to use tiling textures, the end result was they looked the same except the pixel density was higher resulting in crisper textures and I was able to modify the asset into the 6 different variants that would of been difficult with the unique unwrap.

as far as the layered normal maps, I believe the method would be to use a tiling normal map with large macro details on say UV channel 1, then the detailed map on UV 2, combined the tiling is different on both maps removing the tiled look. Personally I've only used layered diffuse maps on WAR, so I haven't done it with normals.

arshlevon · Mar 2010

for stuff like this i just use the zproject brush, i only model one finger and project it onto the other ones.

hijak · Mar 2010

reiro wrote: »

Kinda makes sense but wouldnt that be a lot of different maps? I mean the cave if uved is probably more than one UV Map since its quite big. If you then have a large scale normal and a detail tilable normal isnt that exeeding the budget?

Iam only concerned when you say "this is how people do it all the time" that its not the way it is good for your portfolio. At the end of the day i wanna have a good portfolio, this isnt only done for fun so iam a bit concerned whats "okayish" in industry terms.

I assume every way has its downsides. If you make it tilable you are losing out on lots of unique detail, if you make things too unique you exceed the budget and its not really made for games anymore.

look for the post on how they do the snow in unhcarted 2. its basically a similar idea, and you are saving on budget lemme explain.
so yeah you have one large scale nrm map. maybe its a 1024 or 2048.
well your tiling textures can be as small as 128 if you want, as the combo of both maps on top of each other will give every area a unique look. so its really not much. and whats wrong with doing something a certain way. what matters most is that it looks good when its done. AS long as it looks good and is in a reasonable budget it will be okay, but no one will care what method you used if you cant get the good looking result your looking for etc.

but really they use this in a ton of games, and its a very common way of doing rocks, and terrain type stuff, mixing textures in various ways through shaders has been around for a really long time.
any way should work though, some will just be a lot more work than others.

with the methods we are brining up you get uniqueness at the cost of very little to the budget. becasue you can use the same tiling texture for every rock in the enviro and then just need one nrm map for each unique group. thats just my 2 cents though.

reiro · Mar 2010

hijak wrote: »

look for the post on how they do the snow in unhcarted 2. its basically a similar idea, and you are saving on budget lemme explain.
so yeah you have one large scale nrm map. maybe its a 1024 or 2048.
well your tiling textures can be as small as 128 if you want, as the combo of both maps on top of each other will give every area a unique look. so its really not much. and whats wrong with doing something a certain way. what matters most is that it looks good when its done. AS long as it looks good and is in a reasonable budget it will be okay, but no one will care what method you used if you cant get the good looking result your looking for etc.

but really they use this in a ton of games, and its a very common way of doing rocks, and terrain type stuff, mixing textures in various ways through shaders has been around for a really long time.
any way should work though, some will just be a lot more work than others.

with the methods we are brining up you get uniqueness at the cost of very little to the budget. becasue you can use the same tiling texture for every rock in the enviro and then just need one nrm map for each unique group. thats just my 2 cents though.

Thanks man, really appreciated. This method could work if my rock wouldnt be so big i assume.
As said above the problem is how big will the map be? A whole map for the whole rock would be extremely blurry at 1024 but could be "rescued" by the tiling detail...i ve to try it out to see what works best

ralusek · Mar 2010

by the way, projection in zbrush 3.5 is admittedly botched, but fortunately there's a decent workaround by setting the PA blur to a multiple of 8. so if you just set PA blur to 8 instead of 10, that should stop the artifacts that appear from what would otherwise turn out as a solid projection. that is the only solution pixologic has offered until they address this

reiro · Mar 2010

ralusek wrote: »

by the way, projection in zbrush 3.5 is admittedly botched, but fortunately there's a decent workaround by setting the PA blur to a multiple of 8. so if you just set PA blur to 8 instead of 10, that should stop the artifacts that appear from what would otherwise turn out as a solid projection. that is the only solution pixologic has offered until they address this

Cheers. Good to know will give it a try.

As for the texturing of the rock i ll just use tilable texture and vertex blend in the shader. After looking into it a bit more it seems to give the best results for a thing like this.

Is there a way to create moss, lava or dirt by Zdepth in an UDK shader? Like the snow in Uncharted 2?
Havent found anything on it so far only the vertex paint with a height map as mask.

PS: Is there a way to extrude the edges of a plane to expand the geometry in Zbrush3.5? I want a bit more room around the head sculpt. Have never done it and couldnt found anything on the web so far.

hijak · Mar 2010

reiro wrote: »

Cheers. Good to know will give it a try.

As for the texturing of the rock i ll just use tilable texture and vertex blend in the shader. After looking into it a bit more it seems to give the best results for a thing like this.

Is there a way to create moss, lava or dirt by Zdepth in an UDK shader? Like the snow in Uncharted 2?
Havent found anything on it so far only the vertex paint with a height map as mask.
.

i dont think you would want to do it by zdepth that would be more useful for say fog. For something like snow or moss you are ussually trying to have it either stick to the tops of objects, so would be surface normals in positive Y. And if you mean based of a heightmap, your talking more of a masking based thing, where the heightmap is the mask. Just keep looking around ive seen numerous posts about these various methods. And there is not reason why you cant combine multiple methods as well. using vertex blending and heightmap based masks.

here i think this may be the thread, im sure there are others too.
http://boards.polycount.net/showthread.php?t=68089&highlight=advanced+vertex+blending

Eric Chadwick · Mar 2010

More about vertex blending with a bitmap modulator...
The Snow and Ice of Uncharted2?

SHEPEIRO · Mar 2010

ive been pondering if it was better to use the blue channel of the overlying normal map for material blends like this?

any idea if systems like this use the map space normal for blending or the object x map space (should be fairly easy in a deffered rendering system where this stuff is stored in a buffer anyway)

reiro · Mar 2010

Just wondering if the amount of decimation for the rocks is enough considering its size. The whole rock is about 12k tris which isnt a lot. How high would be the pixeldensity be in next gen games?

Also before i bake the high poly sculpt is it a good idea to set differnt smoothing groups to get harsher and sharper edges?

Eric Chadwick · Mar 2010

Since you're gonna normalmap it, the mesh density only matters where you're going to notice a silhouette. Most of those triangles are wasted. It's not just a lot of vertices to transform, it's also a lot to load into VRAM & to store in memory.

The lighting is going to be static right? Then don't bake a huge normalmap, that's a huge waste of memory, and you'll end up with a blurry mess anyhow. I'd recommend tiling normalmap(s) instead, with a lightmap and maybe some vertex AO.

reiro · Mar 2010

EricChadwick wrote: »

Since you're gonna normalmap it, the mesh density only matters where you're going to notice a silhouette. Most of those triangles are wasted. It's not just a lot of vertices to transform, it's also a lot to load into VRAM & to store in memory.

The lighting is going to be static right? Then don't bake a huge normalmap, that's a huge waste of memory, and you'll end up with a blurry mess anyhow. I'd recommend tiling normalmap(s) instead, with a lightmap and maybe some vertex AO.

So you mean retopo by hand in order to save even more tris?

Well there will be one static sun light and slight reddish light from the ground due to small lava cracks in the ground...
well i ll be using tiling normal maps anyhow. I might just need ONE area where there is a unique "crack", since there is one running through the environment which causes the slight lava cracks in the ground. Maybe i ll use a decal to get the crack going.

Eric Chadwick · Mar 2010

I would just decimate some more, keeping in mind that you really only need enough to define the protruding edges that the player will see. All the recessed detail doesn't matter for squat, since no one will be able to tell it's there, beyond some really minor parallax.

Also if this was for a level in a real shipping game, you would want to split the rock wall into quadrants, like N S E W and top/bottom, so parts can be culled by the engine when they're out of the player's view.

My 2 cents anyhow.

reiro · Mar 2010

EricChadwick wrote: »

I would just decimate some more, keeping in mind that you really only need enough to define the protruding edges that the player will see. All the recessed detail doesn't matter for squat, since no one will be able to tell it's there, beyond some really minor parallax.

Also if this was for a level in a real shipping game, you would want to split the rock wall into quadrants, like N S E W and top/bottom, so parts can be culled by the engine when they're out of the player's view.

My 2 cents anyhow.

I see. Good to know and thank you once more.

yiannisk · Mar 2010

perhaps you would like to also use parallax mapping or similar technique to get more detail out of it?

all it requires is a black and white displacement map and ofc support from the engine..

reiro · Mar 2010

yiannisk wrote: »

perhaps you would like to also use parallax mapping or similar technique to get more detail out of it?

all it requires is a black and white displacement map and ofc support from the engine..

Ah ok, so it works like a "virtual displacement" to get more depth?
Yea i think UDK supports it but iam not 100% sure.

Adam L. Gray · Mar 2010

It does indeed support it; http://aaronjclifford.com/images/tutorials/prallaxBakingDemonstration.jpg

reiro · Mar 2010

Because i dont want to open a new thread for this i ll just post it here.

For some odd reason i dont keep the detail when upping my addaptive skin after the retopo process. it just blurs into one giant mess. Any ideas why Zbrush R3.5 does this? Even when doing denser retopo its not keeping the details.
Never retopo much in Zbrush so it might be some settings which i cant figure out or topo issues.

ScoobyDoofus · Mar 2010

reiro: Are you projecting your mesh details after you retopo? I dont use Zbrush for retopo usually, but I do know that any time I try to transfer data from one mesh to another, I need to perform a projection step. From what I'm seeing, it looks like you may have skipped that.

reiro · Mar 2010

ScoobyDoofus wrote: »

reiro: Are you projecting your mesh details after you retopo? I dont use Zbrush for retopo usually, but I do know that any time I try to transfer data from one mesh to another, I need to perform a projection step. From what I'm seeing, it looks like you may have skipped that.

Yea neither do i. Thanks for that thats the step i missed. Saved me some time there.

reiro · Apr 2010

This is a quick first test just of the normals of the tileable rock texture. It does not look much like rock atm. Is it better for rock to make shapes that are more "connected" with each other and less deep? I tried to keep it small shapes in order to make it look less repetitive. I guess i have to study rock walls a bit more. It seems really hard to make a good tilable texture for a place that big without getting noticable repetetive pattern and still remain the rock look.

I also have quite a lot of seems due to the vast size of the rock.
I would like to have a bit bigger chunks like in Borderlands, so just upscaling the texture? Or redoing a new one with less shapes?

Any hint to improve it or comment is appreciated.

Eric Chadwick · Apr 2010

You do need reference photos, that's for sure. Looks lumpy because the rocks are very evenly-sized and evenly-distributed.

Good reference site.
http://cgtextures.com/textures.php?t=browse&q=349

Some tuts that might help.
http://wiki.polycount.net/Digital_Sculpting#TS

reiro · Apr 2010

Another try.

Please Note: This is just a quick texture which i crazybumped. Its not sculpted. Nor is there a real scepcular on it.
It was just to see what works and what doesnt. Next step would be the sculpt.

Now before i start sculpting something similar to that. Whats good what isnt? Or is it a good base as orientation for the sculpt?

Eric Chadwick · Apr 2010

The texture looks blurry, perhaps not tiled enough. If you're not using a multitexture approach, then I would tile the rock normalmap something like 3 to 4 times vertically from the floor to the roof, and in the horizontal to match so the aspect ratio is 1/1.

It's still gonna be blurry tho from the player's perspective when they're walking right up alongside the wall. Detail texture could help with this, depends on the engine. Which one are you shooting for?

Anyhow, not sure you're looking at appropriate reference for something like this. Reference is crucial! Some ideas...
http://images.google.com/images?&imgsz=l&q=cave+skylight+-webshots

reiro · Apr 2010

Iam using unreal engine.

Well the rock is not directly on the ground there is a wall beneath so the player cant walk close.
You are right its not quite there yet in terms of rock style. Will gather more ref.

monkeyboy_garth · Apr 2010

I find creating your own alphas for use in zbrush helps a hell of a lot...check out cgtextures.com

reiro · Apr 2010

monkeyboy_garth wrote: »

I find creating your own alphas for use in zbrush helps a hell of a lot...check out cgtextures.com

Thanks, yea i ll keep that in mind.

Anyhow, have done different directions of rocks for testing. After looking into reference i figured there is not "one" type of rock so its depends what mood and feel you aim for.

Still not happy with any of them even if its just a texture atm and not sculpted.

The problem iam having is that a tileable rock texture can easy be hard on your eyes if there is lots of details...as seen in pic 1 and 3. However if you have too few eye catching details there is detail missing pic 4.

Now i dropped the type of rocks on 2 since its more like a cliff.
Iam aiming for an "aggressive" kinda looking rock something like this: http://www.a2jlp.co.uk/photos/images/covecave01.jpg

So iam working towards pic 1 with a bit more aggressive forms. Havent thought making rock can be so hard. Any one any idea what kind of rocktype the pic might be?

Eric Chadwick · Apr 2010

Option 1 seems to fit your geometry the best of all four.

Did a quick Google Images search for "rock types"
http://geomaps.wr.usgs.gov/sfgeo/geologic/stories/geologic_materials.html
http://geology.about.com/library/bl/images/blrockindex.htm

Maybe basaltic?

reiro · Apr 2010

EricChadwick wrote: »

Option 1 seems to fit your geometry the best of all four.

Did a quick Google Images search for "rock types"
http://geomaps.wr.usgs.gov/sfgeo/geologic/stories/geologic_materials.html
http://geology.about.com/library/bl/images/blrockindex.htm

Maybe basaltic?

Yea seems to be basalt. Did 3 more types.

I guess i ll stick to type number 1. Thanks for the feedback. Think i research a bit more into that rocktype and get on sculpting.

reiro · Apr 2010

One more

Will sculpt these two and see what works best with the temple style and arch.

Eric Chadwick · Apr 2010

This belongs in P&P now, maybe a mod can move and rename it for you.

#2 looks a lot better, IMO. And I would either map the specular to isolated bits/veins, or turn it down significantly. Looks oily.

reiro · Apr 2010

EricChadwick wrote: »

This belongs in P&P now, maybe a mod can move and rename it for you.

#2 looks a lot better, IMO. And I would either map the specular to isolated bits/veins, or turn it down significantly. Looks oily.

Yep will just use another thread from now on. Does not rly belong here anymore.

How to merge a Sculpt with Rocksurface the right way

Replies