xNormal Hi-Poly to Lo-poly Baking in UDK - Lighting Seams
polycounter lvl 8
I recently ran into a problem where after importing a FBX file of a character I modeled in 3ds Max to UDK, and applying the Diffuse and Normal map, I get nasty lighting seams.
From what I understand, the centered lighting seam is from me applying a symmetry modifier in 3ds Max after unwrapping half the model (I know, it's bad). However, even after extensive searching on this forum on what not, I can't seem to get rid of the lighting seams that occur at the edges of my UV Islands (Apologies if the images are too large, I took these screenshots yesterday, and I didn't bother to resize):
And another with a Tangent Normal Map for comparison:
And, the UV layout:
What's the best way to solve this problem (I use 3ds Max, UDK, Mudbox, and xNormal)?
I've tried using TexTools to generate the smoothing groups by UV Islands and whatnot, then re-rendering the hi-poly over the corrected low-poly, but nothing changed. I may have done this wrong, but I'm self taught at this stuff, so I probably jacked something up.
Thanks in advance.
From what I understand, the centered lighting seam is from me applying a symmetry modifier in 3ds Max after unwrapping half the model (I know, it's bad). However, even after extensive searching on this forum on what not, I can't seem to get rid of the lighting seams that occur at the edges of my UV Islands (Apologies if the images are too large, I took these screenshots yesterday, and I didn't bother to resize):
And another with a Tangent Normal Map for comparison:
And, the UV layout:
What's the best way to solve this problem (I use 3ds Max, UDK, Mudbox, and xNormal)?
I've tried using TexTools to generate the smoothing groups by UV Islands and whatnot, then re-rendering the hi-poly over the corrected low-poly, but nothing changed. I may have done this wrong, but I'm self taught at this stuff, so I probably jacked something up.
Thanks in advance.
Replies
For whatever reason, mirroring the UVs in channel 0 creates lighting seams in static meshes. If this is a character then that's not a problem, since characters are skeletal meshes, not static meshes.
The seams will appear whether or not you're using a normal map, if I recall correctly. Or maybe it was that they only appeared when you toggled on the normal map, but it didn't matter what normal map you were using. I don't remember which.
Looking at Epic's props, it looks like they bypassed the problem by covering mirroring seams with geometry, or by only mirroring pieces that don't directly touch each other (such as tires on a car), or by adding detail normal maps - which make such seams less apparent.
It's not an issue that you can fix, as far as I'm aware. I ran into this same problem a while back and wound up doing what epic (apparently) does - just covered it up.
If anyone's actually found a fix for this, let us know.
EDIT:
just re-checked the image. There isn't actually a mirroring seam, so I am guessing that what I wrote above isn't your problem (but I bet a seam will appear if you bake lighting).
My first guess based on your screenshots would be vertex normals. If you're using Maya, import the fbx (or whatever you brought into udk) and hit 'Normals > Unlock Normals'. If that doesn't do anything (you'll know) then that's not the problem either. If it DOES do something, then fix your normals to however they should be, and re-export.
And make sure that when you export your model, you are exporting vertex normals. It's called something different depending on what file format you're exporting. FBX calls it "smoothing groups".
Actually, you were right at first when you said there was a mirror seam, it's just not apparent from the angle that I took a screenshot of the scene. I read up that using DetailMaps or something could hide the seam, but I'll play with that later.
I don't use Maya, so I wouldn't know where to begin on finding out how to do "Normals > Unlocks Normals". I use 3ds Max for anything that isn't sculpting.
Also, for the person who suggested handplane, I tried that, but all that did was increase the detail of my normal map (to an extent), but more or less amplified the lighting seams.
Sorry, man. Can't help you there.
Different idea for what it could be: if you baked lighting, it could be a problem with that.
UDK should use vertex lighting if you only have 1 UV channel, but just to be sure, give your model some lightmap UVs.
Recompressed Map from Handplane:
Recompressed Map before Handplane:
I'm considering just completely bailing on this mesh, and making simple environmental meshes instead. Thanks for the assistance regardless of whether or not I give up on this one.
The settings in xNormal were X-, Y-, Z+.
It should be
x+ Y- Z+
I meant when he said to invert the Red channel. My original settings were X+, Y-, Z+ to begin with. My bad.
I overlooked that, but placing the mirrored UVs outside of the 0-1 space did nothing different. It looks like it would if I hadn't changed the UVs.
If you could post your lightmap layout that might be a big help