So I know that converting a grayscale image to a normal map is technically incorrect because the converted normal map parts do not actually correctly form to the tangents of your low poly model, but is there any way to make this work? Im assuming that when people do this they are just saying "fuck it" and letting it slide even though it is technically an inaccurate way to add parts to a normal map.
The way I'd want to use this and have used this in the past is by adding textural details, like fabric details in photoshop to the diffuse texture then converting that using something like the NVIDIA normal map filter but Im having second thoughts on ever doing that again since it might not be correct. And I guess this also means that using nDo's normal map creation tools is technically wrong as well. What is the general thought and wisdom behind techniques like this? Is it generally better to be a purist about normal maps or is the conventional wisdom that it just doesnt really matter since it doesnt create that huge of a negative impact on the quality of a normal map?