UE 5.3: Unpacking and Recalculating Normal Blue Channel Issues
Hello, I am looking for advice on how to combat an issue.
Process: Sculpts in Blender -> Baking in Marmoset -> Texturing and Channel Packing in Designer -> Import into UE (Compression Settings BC7, DX11, optional A; Sampler Type: Linear Color)
I've been getting data loss in my normals when trying to recalculate the blue channel. In Designer, I export normal, occlusion, and height in one map (RG,B,A) respectively. Originally they were exported as raw, but I've since read up and made sure to export in linear color space. (The output node is C16, color 16 integer).
I've tried several methods of recalculating the Z normal, and all achieve artifacts with some being closer to the original normal map than others.
1. Is this data loss due to the angles of the sculpt? I've read that when baking steps are taken to avoid missing data in the actual baking process. Is it possible that this data is impossible to reconstruct? If so, what is the next best step? Should I just reserve this method for unpacking only where it happens to work?
2. Is there a change to UE normal calculation from when these methods were used? The sources I've gotten these from were a tad old, but this doesn't exactly mean they're no longer valid.
3. Advice to prevent this issue if possible in the future is appreciated.
Image Order:
Two of the methods used, bottom one being the closest to the original
Original Normal Bake
Best Blue Channel Reconstruction
Process: Sculpts in Blender -> Baking in Marmoset -> Texturing and Channel Packing in Designer -> Import into UE (Compression Settings BC7, DX11, optional A; Sampler Type: Linear Color)
I've been getting data loss in my normals when trying to recalculate the blue channel. In Designer, I export normal, occlusion, and height in one map (RG,B,A) respectively. Originally they were exported as raw, but I've since read up and made sure to export in linear color space. (The output node is C16, color 16 integer).
I've tried several methods of recalculating the Z normal, and all achieve artifacts with some being closer to the original normal map than others.
1. Is this data loss due to the angles of the sculpt? I've read that when baking steps are taken to avoid missing data in the actual baking process. Is it possible that this data is impossible to reconstruct? If so, what is the next best step? Should I just reserve this method for unpacking only where it happens to work?
2. Is there a change to UE normal calculation from when these methods were used? The sources I've gotten these from were a tad old, but this doesn't exactly mean they're no longer valid.
3. Advice to prevent this issue if possible in the future is appreciated.
Image Order:
Two of the methods used, bottom one being the closest to the original
Original Normal Bake
Best Blue Channel Reconstruction
Replies
dont fanny around with color space in designer unless you know exactly what space the textures you are importing are stored.
marmoset will (should) poop out normals, occlusion and height in linear space which means that if you leave designer at it's color managed defaults and export raw it won't break anything
on import into unreal you need to make sure you're not applying an srgb curve to the images.
importing using the masks type should work fine for that
second thing:
I assume you found this while you were digging,
https://discourse.techart.online/t/how-to-calculate-the-blue-channel-for-normal-map/4436
theodox knows his shit so without checking the maths out for myself I'd assume the sqrt(1-(sqrt(r)+sqrt(g))) method would work out (provided your normal map was normalized before extracting the r and g channels)