Displacement map "Surface Dithering" problem

Gaurdo

Hi all!  I would be very grateful for your help!

Situation:

- I bake Displacement Map 32 bit .EXR through any of the listed Xnormal, Knald, Mighty Bake

- At the output I get a map, when applied, dithering (aliasing, ladders) appears

in 3ds max i fixed it with VrayBitmap filtering.

But how to fix it without VrayBitmap, and filter disp. map for zbrush for example ?

Eric Chadwick

This looks like a viewport preview. Bitmaps are downscaled automatically for realtime feedback. What does it look like when rendered?

Gaurdo

Hi there! Thanks for your reply! Unfortunately it has the same condition in maya, zbrush, 3ds max view ports and renders like Vray, Scanline, Arnold. Here is an examples.

But in case of 3ds max, I've found that it changes with different parameters of filtering in a standard max "BitMap" map:

And if i use "Vraybitmap" map, i can manually edit the intensity and chose an algorythms of filtering. And it works great like in a shot below:

But the problem is, that it works only in 3dsmax . And i cant find the way, to filter it in Zbrush. Or maybe there is a way to save a "filtered" version by "VrayBitmap" algorithm out of 3dsmax in a separate file? Or even

Eric Chadwick

Are you sure it’s 32 bits per channel, and not 32 bits per pixel?

Gaurdo

I am not sure, but that what i have when i am open a maps i baked:

Some maps i bake with Xnormals, some of them in Knald, some of them In MightyBake. And all of them has the same problem.

Eric Chadwick

Can you share a small example file? Like a 512x crop of the original.

Gaurdo

I tried to crop it in 512x512 but it became too small. Here is a sample of scene i use to make a tests. There is a simple sphere with UVs and a FullFloat 32bit map non Normalized with a simple relief in it. I apologize for sending the file, because you asked to send the croped map. But if I cut the size down to 512 there is no room for any dispalcing information. That dithering effect appears if sphere mesh subdivided 6kk polys and higher. In other words the higher mesh density - more visible "dithering" effect become.

pxgeek

Looks like you're baking out an rgb texture.

How does baking out a 16bit grayscale look?

I've only baked 16bit tif files for displacement and seem to work well enough for me.

Gaurdo

I tried to bake 16 bit full float with Knald, and the result is even more weird.

Tried to bake with Xnormal 16 bit rgb. Unfortunately there is no option to bake it in grayscale, at least i didnt find it.

Map converted in gray scale in photoshop

Eric Chadwick

Oh maybe because you're using the old Displace modifier? Which renderer are you targeting in Max? If you're using V-Ray, it's better to use the VRayDisplacementMod modifier.

Eric Chadwick

I'm not seeing problems with your ZIP.


This is rendered with V-Ray viewport IPR. I did have to add a Smooth modifier and set it to Smoothing Group 1, just to get nice smooth normals. TurboSmooth is set to 3 iterations.

This is in 3ds Max 2024.2.1, with V-Ray 6.20.02. WIndows 11 Home, NVIDIA GeForce RTX 3080 Ti, 64GB RAM.

What are your hardware specs?

Eric Chadwick

Also what units are you using in Max, and how big is your mesh? I'm using 1 meter, and the OBJ comes in at about 126 meters tall.

Gaurdo

Unfortunately same case with VRayDisplacementMod:

Here is 16 bit TIFF

Here is 32 bit EXR

I am not targeting to use renderers in max.  In that case i just tried to learn how to bake clean displacement maps, to use them as sculpting tool for Zbrush for example. And i made a simple scene with a "Reference Deformed Sphere" with one pressed outside and one pressed inside relief. And the task was to recreate defromations from "Reference Sphere" on a "Target Sphere" by using just a displacement technology. And i thought it would be easy =(

Gaurdo

So in that case. Even if i use VrayBitmap Filtering to fix that problem in  3ds max, i cant fix it in Zbrush. That is why i am trying to understand why that "dithering" effect appears.

And also i am  very grateful to you guys for your support.

Eric Chadwick

Hardware specs?

Gaurdo

Intel Core i7 3770, GTX 1070, 32GB RAM.

Gaurdo

Oh i am sorry i didnt recognize your reply here 12:33AM, really sorry for it. Big thanks to you for your tests. May i ask you to increase Turbosmooth iterations up to 7 or 8 to get sphere 8kk or 24kk? Because it works fine even for me on a low subdivs.

Gaurdo

I use centimeters as units.

Eric Chadwick

Turbosmooth set to 7 is insane overkill, but ok.

I imported the OBJ and reset Smoothing Groups to a single group. I added a Slice modifier to cut away the rest of the model. TurboSmooth at 7. Displace, and loaded your EXR in the OSL Uber Bitmap node. Maybe your import settings need adjustment?

Eric Chadwick

FWIW, here's my MAX file. Just add your "disp map.exr" and turn on the TurboSmooth (off to decrease file size).

Gaurdo

Hello again! I tried to open the file. But my max is to old to make it.

Now i am trying  to replicate the scene by your description, with OSL Uber Bitmap Node.

Eric Chadwick

Which version of Max are you using? I might be able to down-save to it.

Gaurdo

2022

Eric Chadwick

2022 version!

Gaurdo

Huge Huge sorry !!!  Newbie is in action. I gave you a wrong file.

I recorded the clip to explain the situation. Please watch in 1.5x, my English is very bad     https://www.youtube.com/watch?v=FTQO8PvuDtQ

Hope you still interested.

And also here is  a correct file, but i think its already useless.