Baking displacements in Xnormal/Reoccuring flaw/unwanted valleys and seams.
polycounter lvl 7
Hello everyone! For about a year now, I've been obsessed with one particular technical issue with displacement maps. I'm constructing a photo real scene which I've sculpted some beat up wooden boards for in Zbrush. At first, I was importing those maps out of Zbrush with the standard exporting and UVMaster plug-in for UVs. Sometimes I resorted to simple AUV tiles. This works fine and everything rendered out in Mental Ray fine.
One day I decided I wanted to lay out some efficient UVs of my own in 3dsMax and then bake out my displacements from Xnormal. It makes more sense to use all the space available in one 4k map to displace several variants of my wooden boards than have several 4k maps with tons of negative space for each individual board. Besides, the number of map files I've got floating around is really getting out of hand.
The problem here are these unwanted ugly seams/valleys that show along the corners of the low poly model. I do not believe they are caused by the UVs seams because the valleys are showing up where there are no UV seams.
Lets start from the top, this is what the problem looks like.
Here is my UV layout done in 3dsMax.
Here is a close up of the displacement map for the wood board in the example.
The Xnormal settings.
Here are the settings for the tone mapper in Xnormal.
And lastly, here is what I got after attempting to achieve better results by retopologizing and baking.
As you can see, the problem still exists. I've probably rebaked and re-laid out the UVs around a hundred times to try and figure this out. I'm only able to achieve the results I want when I use Zbrush's standard exporting tools. I'm quite exhausted from all this trial and error and I'd be grateful for any advice. Thanks everyone!
One day I decided I wanted to lay out some efficient UVs of my own in 3dsMax and then bake out my displacements from Xnormal. It makes more sense to use all the space available in one 4k map to displace several variants of my wooden boards than have several 4k maps with tons of negative space for each individual board. Besides, the number of map files I've got floating around is really getting out of hand.
The problem here are these unwanted ugly seams/valleys that show along the corners of the low poly model. I do not believe they are caused by the UVs seams because the valleys are showing up where there are no UV seams.
Lets start from the top, this is what the problem looks like.
Here is my UV layout done in 3dsMax.
Here is a close up of the displacement map for the wood board in the example.
The Xnormal settings.
Here are the settings for the tone mapper in Xnormal.
And lastly, here is what I got after attempting to achieve better results by retopologizing and baking.
As you can see, the problem still exists. I've probably rebaked and re-laid out the UVs around a hundred times to try and figure this out. I'm only able to achieve the results I want when I use Zbrush's standard exporting tools. I'm quite exhausted from all this trial and error and I'd be grateful for any advice. Thanks everyone!
Replies
2. I suggest you to model your HP mesh using the LP as subdiv0 level. In that way you can use the MatchUV feature in xN and avoid to setup cages.
3. To avoid problems at edges, sure you setup well your smoothing groups and plan the UV seams ( for instance, you could set all the seams on the botton part which is not visible ).
If you use a cage sure it's continuous and control the direction to avoid wavey effects. If you break the cage using vertex normals you'll get hard edges baked.
3. A bucket size of 256 is enourmous for CPU rendering, that will only hurt performance. Use 32x32 or 64x64 as max.
4. xN 3.17.8 is prehistoric :poly136:
Hahah, I didn't realize my Xnormal version was so out of date!
Quick question, can you explain what you mean by "break the cage"?
ExcessiveZero, I think my UV layout I posted is confusing. I didn't unwrap the high poly. All those other UV shells you see are other wooden boards I've sculpted and some of them I imported into max at a mid level subd level from Zbrush. Thanks for your help.
If your normals are averaged the cage will be continuous.
On the contrary, if you used different smoothing groups for each face ( which occurs in a cube ), the cage won't be continuous:
For organic models you usually want a continous averaged cage.
For non-organic models like a weapon, industrial parts, etc... you may want to break the cage near the edges.
Anyways, you can break/weld the cage's vertices in the 3D viewer ( or to define an external cage or use the "harden normals" option, etc...).
Judging your screenshots your cage is not continuous ( our you're using uniform ray distances ), that's why you get those strange hard edge artifacts in your wall's edges. Setup a cage and make sure it's continuous.
Over the past three hours I've had some degree of success with this. I beveled the harsh corners and then subdivided that mesh. It was then used in Xnormal as the low poly model. This worked pretty nicely. I do still have some minor seams. See below.
At this point, I'm a little torn between using this troublesome method to maintain control over my UV layouts or simply exporting out of Zbrush and not bothering with elegant UVs and neatly organized project files.
Anyone else have an opinion on this?