Photoshop showing 32bit Normal maps from Marmoset washed out
polycounter lvl 2
I've gone through quite a lot of threads on polycount that seemed very promising but none of them anwsered one of the seemingly most important questions in regards to washed out normal maps.
My goal summary: 32bit normal map in Photoshop should not be washed out
These two images are are the same normal map in Photoshop, the left one baked with 16bit and the left one with 32bit.
My goal summary: 32bit normal map in Photoshop should not be washed out
In Marmoset 4, baking a 16bit normal map as a PSD file, will open normally in Photoshop. When baking a 32bit normal map as a PSD however, it's a lot brighter when opened in Photoshop.
I'm aware that this does not affect the file and when loading the normal map into Substance Painter or other 3D software one simply disables sRGB or sets it to linear/ non-color, which will give it that nice blue again. But how do I have the 32b and 16b normal map look the same in Photoshop. I tried using different ICC profiles and I do think that using the correct ICC profile is the key to fixing this, but until now I've had no luck in finding what the correct profile would be. I downloaded this Rec 709 ICC profile (https://www.color.org/rec709.xalter) which Blender uses for it's linear color profile, but in Photoshop it didnt't make the 32b normal map look like the 16b.
These two images are are the same normal map in Photoshop, the left one baked with 16bit and the left one with 32bit.
Replies
The text below I wrote before coming to this realization, but I'll keep it for those that come here with the same issue and need some extra explanation / context.
---------
---------
Adding an adjustment layer isn't a great solution as I have to add it to every file I work on, and it makes the Photoshop files themselves not usable anymore as I would constantly have to disable this gamma over correction layer for when a software is still using that PSD file. (And yes 0,45 is the correct value, thanks for that.)
I also had tried converting the PSD file to 16bit but then when I used "Export As" PNG with "convert to sRGB" off, all of a sudden it did not fix the normal map that was exported, and it was washed out. I tried changing the bit depth with "don't merge" initially which resulted in the issue I just mentioned.
(image of pop up window when converting from 32b to 16b with "merge" option)
I'd advise that if you're doing anything with normals or other 'maths' textures (roughness etc) as photoshop will apply (and save) transforms, you won't know about it and suddenly what you thought was a value of 127,127,127 will actually be 141,141,141
photoshop isn't doing anything objectively wrong apart from not warning you about it - its just not aimed at working on images that want to be manipulated in linear space .
If marmoset is applying an sRGB transform to the values at export (this is unlikely) you need to resolve the issue there, not in photoshop
to determine whether that's the case or not, open the file in something like substance designer or an image viewer other than the default windows one
Obviously 32 is total overkill for the final result (in game engine or render). But as you might know, when baking especially with normal and AO maps, one often gets nasty bading artifacts which are most visible on flat surfaces. Baking the images at 32bit will most of the time fully remove banding and other such artifacts. After baking one can then change the bit depth back to 16bit.
I was trying to postpone the conversion to 16bit to the last possible moment as working on a file with more data doesn't hurt (the entire purpose of EXR files), but as you mentioned Photoshops tools are quite limited in 32b mode, things like heal brush and even fill buckets just don't work.
Another theoretical reason why switching the bit depth at the last possible moment (for most people that's probably during Substance Painter's texture export) is better, is because one should try not to switch between bit depths to often (the same goes for switching between file formats). Switching from 32 to 16 and the deciding one wants 8 might yield worse results than doing one single conversion from 32 to 8.
But working with 32bit files is as of 2024 just not very comfortable, so in conclusion I would say the fix is to bake in 32bit and then immediately convert it to 16bit.
I like photoshop or now Affinity Photo for fixing mistakes in the AO maps. I always frist try to sort out issues with topology and shading frist, but at some point it's just small little things that can easily be fixed in a image editing software. Also if you have ever had the topology grid pattern appear in the AO map due to the low being too low, then a good fix (that doesn't involve editing the low) is to render to AO maps one with the grid fixed (by adding a temp SubD) and one without and then fix both of them together, to remove the grid pattern, but preserve the correct AO map arround the edgeds of the model.