The ever divisive problem of terrain creation.
polycounter lvl 2
Hey polycount community,
I've heavily researched these forums looking for a solution to my struggles in ZBrush, however I haven't found many instances covering this specific subject, so incase I had omitted some topics here that would help me, please redirect me to them and thank you! 
I'm dealing with what appears to me as a really complex situation. I've modeled a large piece of terrain (currently in a primitive state) that features some very steep and vertical areas. These areas are designed to be cliffs. As is the case with height maps or pretty much any primitive terrain, these areas are extremely simple and plain. Therefore, I've decided to go into ZBrush to sculpt some blocky cliff details onto them so they look believable and 3D. This is where I ran into a series of issues that have completely messed up my sleeping schedule lol.
I'm dealing with what appears to me as a really complex situation. I've modeled a large piece of terrain (currently in a primitive state) that features some very steep and vertical areas. These areas are designed to be cliffs. As is the case with height maps or pretty much any primitive terrain, these areas are extremely simple and plain. Therefore, I've decided to go into ZBrush to sculpt some blocky cliff details onto them so they look believable and 3D. This is where I ran into a series of issues that have completely messed up my sleeping schedule lol.
I'll try to cross-explain two main problems I'm facing:
1. I'm fully aware of the possibility of creating separate, modular rock / cliff assets and placing them into / onto the terrain through level design. This of course is what is most commonly done. A huge advantage to this is the simplicity of the whole process, however I believe it has way too many disadvantages that work against it, such as an extreme waste of unused geometry and the ever increasing total amount of assets required to load the entire level. Another downside to this process are seams which obviously point out the cuts between terrain and props. Speaking of seams, using level design to place assets of such kind would make it unimaginably difficult to create decals for in order to hide these seams, especially when you don't have an uninterrupted flow of geometry that is connected/stitched together.
1. I'm fully aware of the possibility of creating separate, modular rock / cliff assets and placing them into / onto the terrain through level design. This of course is what is most commonly done. A huge advantage to this is the simplicity of the whole process, however I believe it has way too many disadvantages that work against it, such as an extreme waste of unused geometry and the ever increasing total amount of assets required to load the entire level. Another downside to this process are seams which obviously point out the cuts between terrain and props. Speaking of seams, using level design to place assets of such kind would make it unimaginably difficult to create decals for in order to hide these seams, especially when you don't have an uninterrupted flow of geometry that is connected/stitched together.
So here I was, thinking to myself "you understand the process of creating modular assets in ZBrush, just apply the same technique here with the terrain". Then the realization hit me. The terrain is a single sided object, which immediately eliminates the use of Dynamesh. So I went over to ZRemesher to save the day, which it did, kinda... I split all my cliff areas into separate polygroups and ran them through ZRemesher, which retopologized them all in such a great way that I was finally able to put some detail on it, until I ran into the problem of messing up the topology myself from all the sculpting work and desperately wanting Dynamesh to fix it for me so I can continue. Seeing as Dynamesh wasn't an option, I went with ZRemesher again. Poof, all detail gone, melted... no matter what I did and what settings I put.
Plan B. Make the terrain a huge closed object by extending the boundary loop along the z-axis and filling in the bottom hole. I should be able to use Dynamesh now! I did, until it gave me the same kind of results ZRemesher did. Increasing the resolution of Dynamesh was also a failed attempt as ZBrush would eventually give up on me for setting it too high in order to retain the sculpting detail I wanted.
Which leads me to the second problem:
2. Plan C was to use previously made modular cliff assets and append/merge them into the terrain the same way you would while level designing, except this time, it would all be done in ZBrush, so once I would place them all as I'd like, I'd hit Dynamesh and it would magically merge the geometry together, stitching the visible geometry of my cliffs with the visible geometry of my terrain. I mean... I don't know what I was expecting... Heck, it did the job, but it completely melted my cliffs to the point of needing to be resculpted.
I have to be doing something wrong. I feel like all three: A, B and C plans have flaws in them and I have to be lacking something in the process of executing them. I'm definitely not as experienced with ZBrush as a lot of you guys are, which is why I'm literally crying for help to you, but I believe I had really tried everything I have knowledge of at this time. If someone could be so generous to point me in the right direction or give me an advice, tip, suggestion or anything to understand how I could do this or expand on some of my plans/attempts/ideas and get me back on track, it would mean everything to me.
Which leads me to the second problem:
2. Plan C was to use previously made modular cliff assets and append/merge them into the terrain the same way you would while level designing, except this time, it would all be done in ZBrush, so once I would place them all as I'd like, I'd hit Dynamesh and it would magically merge the geometry together, stitching the visible geometry of my cliffs with the visible geometry of my terrain. I mean... I don't know what I was expecting... Heck, it did the job, but it completely melted my cliffs to the point of needing to be resculpted.
I have to be doing something wrong. I feel like all three: A, B and C plans have flaws in them and I have to be lacking something in the process of executing them. I'm definitely not as experienced with ZBrush as a lot of you guys are, which is why I'm literally crying for help to you, but I believe I had really tried everything I have knowledge of at this time. If someone could be so generous to point me in the right direction or give me an advice, tip, suggestion or anything to understand how I could do this or expand on some of my plans/attempts/ideas and get me back on track, it would mean everything to me.
Thank you all in advance.
Replies
If it's a game engine, modular cliffs are still the best bet.
Hidden geo where it penetrates the height-map terrain, or other modular pieces, is not that big of an issue performance wise. Unless you;re doing crazy things, like hiding most of the asset and just showing a tiny tip.
If you're having trouble with seams, there are alternatives to the old method of covering them with smaller assets. Quixel has a great post here outlining several methods in Unreal.
https://quixel.com/blog/2020/1/22/blending-megascans-assets-in-ue4
It is indeed a playable level for a smaller game. I've seen many of these blending techniques done in UE4 and I just wish we had the same kind of methods at disposal, but sadly it is not the case as we're not using UE4 or Unity. The way we do it is through decal placements over the intersecting seams. I haven't experienced this headache before as most of the sculpting detail wasn't so extreme and needy as is the case now where I'm required to really dig in and apply some strong cuts and creases across the board. It's the vertical areas that are making this so much harder. I'll keep researching and trying out more things. Thanks again!