I'm horribly inexperienced with normal mapping and I'm not the most active polycounter on the planet, so please forgive me if this is an easy fix or has been addressed before.
I've messed around with normals a bit in the past but never too much. The reason being I always run into problems with getting normals to come out looking good in 3D Studio with Xoliul Shader 2. Today I used the 3d Viewer in Xnormal for the first time and got the normals to look perfect in no time. You can see this in the first image. However, in the second image, that same map, applied to the same model, with the same smoothing groups, ends up with a result where the flat sections look concave, and there is a noticable seam down the middle that doesn't appear in the 3D viewer.
Initially I made this model and used Xoliul in 2013, after some research it seemed that this problem is unique to 2013, but I saved the model as a 2012 file and opened it in that version, but that did nothing. Both with the Nitrous shader disabled (As you can see in the screen) I disabled Gamma correction and that fixed the seam, however did nothing for the rest of the model and just messed my lighting up.
Has anyone else encountered this and can anyone help me out?
Replies
What are your smoothing groups? A single one for the entire thing? It looks like you're just getting tangent space problems because your smoothing is too extreme.
Also, for Gamma correction, make sure to change the number for the correction to 1.0 before you turn it off, it's bugged in the sense that it will still sometimes try to use that number for correcting even though you turn it off.
Yes, all my geometry is on one smoothing group in both of those images, I don't understand why it would come out looking good in Xnormal and not in 3ds though, I exported the smoothing groups with it. If I remember right I've tried breaking up smoothing groups on other models but run into similar problems in the past, I'll try breaking them up here to see if that does anything and I'll post the results later today.
I messed around with the smoothing groups and the result is better but still not perfect. The first two images show areas that are good and areas that are bad:
The next two show UV layouts and smoothing groups.
EDIT: I believe I got this fixed in 2012 at least. I was using an old version of the shader from some time ago. I simply reinstalled the newest version and now there doesn't seem to be any problems in 2012.