Normal Map (TS) - Specular/Gloss artefacts

PetSto

Hi polycomunity,
I have problem with specular/gloss on my baked normal map. I have some artefacts there, as you can see on image. I simply export from Maya my LP and HP meshes, both are triangulated, LP have smooth groups and normals exported. In xNormal I set Cage and bake classic normal map in tangent space. Then I inport to Marmoset Toolbag and in material settings I set gloss ad specular intensity to 1 and this is my results.



xNormal - Bake (exported to Tif 8/16/32)
RGB/8 - bad
RGB/16 - good
RGB/32 - good
RGB/16 to RGB/8 - not bad/not good (noise)

But I cant use RGB/16 because I have plan use mesh in game engine (UE4) but there will be all my maps compressed. (I use nVidia DDS plugin to compress)

- RGB/8 compressed to DXT1 / DXT3 / DXT5 and 3dc - bad results
- RGB/16 compressed to DXT1 / DXT3 / DXT5 and 3dc - bad results
- RGB/16 compressed to RGB/8 and then compressed to DXT1 / DXT3 / DXT5 - bad results amd 3dc not bad but not good too, there are some noise



Full size PSD

artquest

This is a known issue across multiple game engines (it's not really any engine's fault though, there just isn't enough data in an 8 bit image to properly represent the normal vectors). Your best option is to bake a 16 bit normap map and convert it to 8 bit. Once you put the rest of your textures in it wont be as noticeable.

JedTheKrampus

You should import your 16-bit normal map into UE4, set the compression preset to TC_Normalmap, bump up the roughness of the material a bit, and celebrate your inevitable success with a horchata. There's no use worrying about this particular issue so much and there's certainly no use bothering with the Nvidia Photoshop plugin to compress these things. Just let UE4 do it.

artquest

JedTheKrampus wrote: »

You should import your 16-bit normal map into UE4, set the compression preset to TC_Normalmap, bump up the roughness of the material a bit, and celebrate your inevitable success with a horchata. There's no use worrying about this particular issue so much and there's certainly no use bothering with the Nvidia Photoshop plugin to compress these things. Just let UE4 do it.

I almost forgot about the automatic compression. Follow these steps and your stuff will look as good as it can currently (LOL)!

beefaroni

JedTheKrampus wrote: »

You should import your 16-bit normal map into UE4, set the compression preset to TC_Normalmap, bump up the roughness of the material a bit, and celebrate your inevitable success with a horchata. There's no use worrying about this particular issue so much and there's certainly no use bothering with the Nvidia Photoshop plugin to compress these things. Just let UE4 do it.

Ya. I made this mistake a lot when I started. Don't bother fussing with small small small normal map errors. Just keep arting

PetSto

JedTheKrampus wrote: »

You should import your 16-bit normal map into UE4, set the compression preset to TC_Normalmap, bump up the roughness of the material a bit, and celebrate your inevitable success with a horchata. There's no use worrying about this particular issue so much and there's certainly no use bothering with the Nvidia Photoshop plugin to compress these things. Just let UE4 do it.

Hi Jed,
ofc I let UE4 compress my 16 bit textures, I only compress textures in nVidia DDS plugin for this example and how and which format is best or no. In UE4 we have some compression settings, but i dont know which use BCn/DXTn compress method (lack of source information on UE wiki). This is list:

TC_Default
TC_NormalMap
TC_Masks
TC_Greyscale
TC_Displacement
TC_VectorDisplacement
TC_HDR
TC_UserInteface2D
TC_Alpha
TC_Distance Field Font
TC_HDR Compressed
TC_BC7

DXT1/2/3/4/5 (BC1/2/3/4/) use DX9 + 3dc/BC5
BC6H and BC7 use DX11

almighty_gir

One setting you may not be aware of (in xNormal), is in plugins - Tangent space calculator - mikktspace: check "Calculate Binormals in Pixel Shader". As this is how both Toolbag2 and UE4 handle their tangent space normals (when in the case of xNormal/mikkt for Toolbag2).

JedTheKrampus

TC_NormalMap uses BC5 compression.

PetSto

almighty_gir wrote: »

One setting you may not be aware of (in xNormal), is in plugins - Tangent space calculator - mikktspace: check "Calculate Binormals in Pixel Shader". As this is how both Toolbag2 and UE4 handle their tangent space normals (when in the case of xNormal/mikkt for Toolbag2).

Wow, this is something new for me, we learning every day as I see But I need more deep explain this function, correct me if I'm wrong. This function is only for types of engines (etc UE4) which can calculate binormals in shader then isn't baked in xnormal and not compresed if I choose TC_Normal Map (3dc/BC5) right ?

Tangent is X in UV and Binormal is Y in UV coordinate system, and normal iz Z (deep). Then we have only 2 channel to store data, X (binormal) and Z (normal) in this case is Red channel birnormal and Blue channel as normal vector. Then we missing data from Y axis (binormal) and when I bake normal in xNormal with this option on I dont have any data from Green Channel then isnt need for store when compress to 3dc/bc5/tc_normalmap where is only store 2 channel to 8 bit, but somewhere I read this compression only store X nad Y, but Y is binormal, I am confused now.

JedTheKrampus wrote: »

TC_NormalMap uses BC5 compression.

Thank you Jed.

JedTheKrampus

PetSto wrote: »

Wow, this is something new for me, we learning every day as I see But I need more deep explain this function, correct me if I'm wrong. This function is only for types of engines (etc UE4) which can calculate binormals in shader then isn't baked in xnormal and not compresed if I choose TC_Normal Map (3dc/BC5) right ?

It's just a slightly different way of calculating the tangent space, or more specifically of interpolating the tangent space between the vertices. The xNormal viewer needs the box unchecked for the tangent space sync to be perfect but every other renderer that uses Mikktspace needs the box checked, including Blender, UE4, and Toolbag. It would be a good thing if the box were checked by default IMO. It's not a huge difference but it's enough to be annoying if you obsess about this stuff.

CreativeHD

This just saved me from so much frustration. I have been baking using 8 bit targas and getting tons of artifacts. I thought it was problems in my high poly not the bit depth. I will have to rebake and see what I get. Ahh beginner problems.