question about "fixing" normals in photoshop
polycounter lvl 12
So I have been told don't fix normals in Photoshop by painting them. But I have had a hard time finding any documentation to support that argument. now I look on the wiki and find:
"Don't be afraid to edit normal maps in Photoshop. After all it is just a texture, so you can clone, blur, copy, blend all you want... as long as it looks good of course. Some understanding of the way colors work in normal maps will go a long way in helping you paint effectively."
Does anyone have a good source of information talks about why I shouldnt fix normals by painting them.
"Don't be afraid to edit normal maps in Photoshop. After all it is just a texture, so you can clone, blur, copy, blend all you want... as long as it looks good of course. Some understanding of the way colors work in normal maps will go a long way in helping you paint effectively."
Does anyone have a good source of information talks about why I shouldnt fix normals by painting them.
Replies
Fixing normals by painting shouldn't be required if you've got your normal baking process down, and sometimes fixing them by painting is misguided. For example, painting to get rid of curves in your normal map will probably make your normals look bad from all angles rather than bad from one angle.
The biggest problem with fixing normals by painting, though, is that it ruins an automatable pipeline. If you update your lowpoly or highpoly mesh or the UVs and want to bake new normals, you're now stuck having to repaint the same fixes.
Yep all of this +1. This is especially problematic if some other poor sap has to come in later and try to figure out the voodoo magic you used to produce a clean bake. Its far better to research and understand the baking process and why typical errors occur (most can be fixed with better geometry in your lowpoly). Plus, it generally takes a lot longer to get a quality result if you rely on hacks and painting over the normal, ao, etc later than to simply spend the time creating a nice clean low that bakes correctly. These days, triangle counts are much less of a concern, so there really isn't the need to create super low poly excessively optimized meshes (which generally will give you more problems when baking) and most problematic areas can be sort out by using an appropriate amount of geometry.
There are quite a few adjustments, like level/curves/contrast that you really simply can not perform on a normal map, especially one that is baked to compensate for the mesh normals of a low poly. A baked normal map essentially contains very specific math to compensate for the smoothing of the lowpoly to make it look like a high poly, even small adjustments can throw this off and result in smoothing errors and other artifacts.
The "do whatever you want" to a normal mentality is from a long ago time when few people actually understood what normal maps did or exactly how they work. Various tricks and hacks and workflows were developed because people didn't understand the root cause of common problems. You actually can do a variety of edits in certain situations, but you really should understand exactly what you're doing first.
More here: http://www.polycount.com/forum/showthread.php?t=81154
and here: http://www.polycount.com/forum/showthread.php?t=107196
Having said that, its still "OK" to paint on it, if this is what you are getting at. It isn't as if dropping a smudge stroke will make the map not read or anything like that. I still smudge out a few small errors that I simply missed for time's sake if it won't be noticed. Also small piece normal mapped details can sometimes be so small that it won't be noticed if you simply delete the errors out by filling the space with the background blue color. This would be like, 5-10 seconds of editing that won't be noticed, versus a rebake. Even then these are mistakes that could have been avoided initially.
I cant disagree with whats been said above.
You definitely shouldnt be fixing bake errors in photoshop (missed rays etc..)
However
If you're building a crap load of modular hard surface pieces that share various bits of your normal map then you will need to make tweaks in photoshop to support it - even if its just making sure an area is actually flat.
Similarly if you're working with heavily compressed maps then somethig as simple as flattening an area or cleaning up a gradient can eliminate or significantly reduce compression artefacts
Also - compositing high frequency detail is much easier than sculpting it in- its also reversible
If I can get 95 percent of the way there with the bake, then fix the rest real quick by hand....I find that to be less troublesome and time consuming than going back and figuring out whats wrong with the bake, fixing it, rebaking, etc.
That being said...if your bakes are coming out really bad where you constantly have to fix things...then the other guys are right and you should probably iron your baking process out better.
But at the end of the day...its just a map full of info, it doesn't matter if its from a bake, from a photo filter, from crazy photoshop actions like what NDO does, or if its modified by hand...all that matters is its correct...or at least correct enough that it looks good.
maybe under a deadline I would resort to it.
Painting out errors = ignorant or lazy.
Painting IN details = great!
Fix your model to fix your bake errors. You can achieve a perfect bake, you just have to WANT to...