Normal Map Issues

Unekorn

I've spent days trying to resolve these problems so i figure if anywhere has any semblance of a solution, it's polycount.

I'm a student artist using Second Life as a platform since it's easy to collaborate on.

Anyway, the problem: (picture speaks a thousand words)



(uploaded image is a JPG for forum friendliness, i realize this isn't a good format, original is a 24-bit TGA)
As you can see the normal map has a sort of lumpy/bumpy consistency when applied to the model. I've done a lot of research into this and discovered it's not an altogether unusual problem when it comes to tangent space normal maps.


(same thing in 3ds max)

(same thing in marmoset)
note: i'm aware of the various other errors in the map, the model/uv has been hacked up so many times trying to make this work i'm not concerned about those just now.

Now, the issue doesn't seem to exist in any software i've tried it in (i also tried it in blender and the unreal engine), in all of these packages the normals produce this seemingly completely smooth/flat surfaces which is expected

Some research tells me this problem is caused by something called Tangent Basis (though i might be wrong), in which each software has a slightly different approach to reading normal maps.

I tried numerous methods of creating the normal maps after discovering this as listed:
(all of the following methods were tested with both Averaged Surface Normals and Exported Surface Normals)
- Baked from 3ds Max
- Baked from Blender
- Baked from Xnormal with both MikkSpace and Unity Space as the calculators
- Baked from Normal as an ObjectSpace in both MikkSpace and Unity Space mode, then Handplaned into every  possible configuration Handplane offers.
- Baked from 3ds Max as an ObjectSpace then handplaned into every possible configuration Handplane offers.

This resulted in about 50 different normal maps in the end, and while some were completely broken about 80% of them had at most very slight variations on the seams/surface.

I went so far as to try digging through the source code for the viewer itself, honestly i've no idea about programming, i dabbled a few hours in C# once at most this lets me identify what the code looks like.
I did actually find the various parts of the source code that calculate the normal, but they seemed fairly standard stuff as far as i could tell, comparing it to the source code for the Unity 3D calculation plugin for xNormal (i can dig that up again and post if someone feels they could make a better judgement on that).

I found an unconfirmed suggestion that Second Life tangent basis is similar to that of Unity's, though honestly i saw little real difference between MikkSpace, Unity and whatever 3ds Max natively produces.

I also found an unconfirmed suggestion that Second Life does not upload the 'Tangent Information' which is apparently stored inside the FBX/DAE files which allows software to correctly display normal maps, which as such might make it actually impossible to have a correct normal map on Second Life.

As you can imagine after days of fudging with this and what feels like a hundred different normal maps, and scowering the internet for answers, i'm starting to think there is literally no solution to this problem, i'm the frustrate able kind of person who wants to try everything, never give up and never ask for help. but desperate times force my hands.

tl:dr: please fix normals!

Scruples

It looks like your model needs to have it's tangents rebuilt in whichever importing tool you're using (or use a format/tool which doesn't destroy the tangent information), you might have better luck on a second life forum.

Unekorn

Scruples said:

It looks like your model needs to have it's tangents rebuilt in whichever importing tool you're using (or use a format/tool which doesn't destroy the tangent information), you might have better luck on a second life forum.

Yeah, the importing tool is very limited on that front, it has no real options in relation to that, the software only supports the DAE format which shouldn't be a problem either way.

Today I tried exploring compression problems, apparently the viewer uses jp2000 as a lossless compression format, which allegedly shouldn't cause any problems but I tried the various range of texture resolutions from 64-2048 and using TGA, PNG and BMP formats prior to upload in case somehow the viewer was losing something in translation there. (no change in any event)

Given the issue seems to more or less mirror the problems seen when using the wrong tangent basis in other engines I'm somewhat certain it is at least related to that, but I've no idea how to go about confirming that.

In the Source Code for the viewer it gives credit to this webpage: http://www.terathon.com/code/tangent.html and informs that much of the code is 'based' on this method, not sure if that's relevant, the full source code is publicly available if anyone has time and expertise on this topic to look at that; it's C++ and the normal map related information is spread over 3-4 files totalling a few hundred lines of code.

The Second Life forum had several people bringing up the topic historically but no one ever really had anything so much as advice, never mind any semblance of a fix or solution, given it's more of a game/user forum than a specialized creative one, and definitely doesn't have the level of expertise/intuition you see on this board, Honestly I'm not sure the issue can ultimately be resolved but i felt if there was anywhere on the internet someone would know, it's probably here.

Raane

You got the green up/down correct for the program?

Unekorn

Raane said:

You got the green up/down correct for the program?

The software uses X+, Y+, Z+, but yes I've tried every combination of -'s and +'s in there, as well as flipping individual islands as a long shot.

Scruples

It looks like most people are using blender to export their models to second life, I know you said you tried baking in blender, but if you exported the dae from 3ds try blender or vice-versa if your exported originally from blender.

Unekorn

Scruples said:

It looks like most people are using blender to export their models to second life, I know you said you tried baking in blender, but if you exported the dae from 3ds try blender or vice-versa if your exported originally from blender.

Honestly I hadn't tried exporting the DAE from different softwares, since the model is triangulated i didn't really think it'd have any effect, as it imports into various other softwares flawlessly, but i can't really say I've tried everything until I have done so.
Exported from 3DS as DAE
Exported from 3DS as FBX then converted to DAE
Exported from 3DS as FBX as legacy (2009) then converted to DAE
Exported from Blender as DAE
Exported from Blender as DAE in "Second Life" Mode.

There didn't seem to be any visual change with any of these tests.