UDK, xNormal and tangents : what the hell is wrong ?
greentooth
Is somebody able to explain me why this happen :
I'm using Maya x64 2013 + FBX 2013.3, and also UDK November 2012. My normal maps are baked inside xNormal (3.17.16).
-When I import my static meshes I of course check "Import tangents" and "Explicit normals".
-When I export my meshes from maya as FBX files I of course check "Tangents and Binormals" in the options.
-My meshes are triangulated in Maya before export. No history on them, everything is clean.
-My normal map baked in xNormal is with the option "use exported normals"
From what I see, it looks like the "import tangent" option is inverted.
I'm using Maya x64 2013 + FBX 2013.3, and also UDK November 2012. My normal maps are baked inside xNormal (3.17.16).
-When I import my static meshes I of course check "Import tangents" and "Explicit normals".
-When I export my meshes from maya as FBX files I of course check "Tangents and Binormals" in the options.
-My meshes are triangulated in Maya before export. No history on them, everything is clean.
-My normal map baked in xNormal is with the option "use exported normals"
From what I see, it looks like the "import tangent" option is inverted.
Replies
>>-My meshes are triangulated in Maya before export.
Plus Freeze transformations.
FBX can be problematic sometimes. Try with OBJ or SBM better.
Also my normal map is flipped correctly.
My transformations are always frozen too.
Really, here the problem is UDK, not xNormal.
Here is the result, disabling "explicit normals" produce the equivalent of "averaged normals" everywhere, the problem is still here and my normals are not imported correctly (obviously, because I disabled "explicit normals") :
[edit]I can see your screenshots now. It's definately the FBX problem. If you can use ActorX that would be the easiest option.[/edit]
Make sure you are using these settings on your texture.
I'm aware about this FBX 2013.2 problem, but the 2013.3 should have fixed this issue from what I have read. unfortunately I'm not a licensee, just an honest user of the free UDK version.
Does ActorX support custom/explicit normals ?
And ActorX doesn't exists for Maya 2013 (an unofficial version yes, but not made by Epic).
To be sure with everybody : my textures are correctly imported.
That's what the guy from Epic told me about a week ago. But I was able to use the code fix, which uses the exported normals to recalculate the tangents durring import.
I assume ActorX supports it, because that was the alternate solution I was given by Epic. I've never tried it though.
The only other workaround I can think of (that I haven't tried) is to export OBJ, then download an older 2011 or 2012 FBX Converter.
http://usa.autodesk.com/adsk/servlet/pc/item?siteID=123112&id=10775920
Looks like a dead end.
[EDIT] I just tested the ASE exporter and from what I remember nothing as change : normals and tangents are not exported with the ASE format for Maya. So UDK recreate them (and they are wrong).
The ASE exporter of 3DS Max is different because this one is native, not handle by a plug-in. The one in Maya with the ActorX has always been limited (this is why I quickly jumped onto the FBX format).
But not tangents/binormals.
Yeah, I tried OBJ and DAE in the FBX 2012.2 converter and it still doesn't work.
The problem with this workflow where you pass a bunch of normals/tangents between applications is that it makes it very difficult to tell what is really happening or where the error originates.
I cant remember what the coordiante swizzles are for each of these tools but it looks like unreal is inverted on one channel compared to maya(or xnormal) which is causing the flip.
This is the problem I'm facing and...
... it's not a swizzle error, my normal maps in UDK are correctly set for X+ Y- Z+.
Passing tangents and normals between applications can get messy and confusing very quickly. Also, keep in mind that this workflow isn't useful for anything rigged- Which includes guns, vehicles, complicated props...
Or do you suggest to bake without the tangent and let xNormal calculate them with the algorithm of Mikk (Mikk - TSpace in xNormal) ? If so, why Epic suggests to import our tangents/binormals if we don't use them to bake ?
I'm just following the pipeline as mentioned here :
http://udn.epicgames.com/Three/XNormalWorkflow.html
Yeah, my current tests were with static meshes only. I'm aware that the Unreal Engine 3 doesn't read our tangents with the skeletal meshes.
After some digging it looks like the current FBX exporter has some bugs, but what is weird is that older versions like FBX2012 used to work fine. I'm going nuts :poly127:
FYI, I'm mirroring wheels and some minor details on a vehicle from one side to another.
Acting on a hunch because I remembered a bug with the "Flip normal map green channel" option box not remembering its settings, I re-imported my mesh with "Import Tangents" turned off but kept "Explicit Normals" on. Guess what? My binormals import correctly and everything is peachy.
I set up a quick test case in Maya with dummy geo and reproduced the issue. I checked my UDK version and I was on the July build. Downloading November now to see if I get different results.
:Update: Tested the newest November Beta 2 build. Using my normal workflow of enabling both Explicit Normals and Import Tangents gave me even worse results! And again, disabling Import Tangents but leaving Explicit Normals on gave me the results I'm looking for.
Can anybody verify? I'm going Maya 2013 -> FBX vers 2013.3 -> UDK
Thanks for the verification cptSwing.
I'm glad to see some other people get the same results as me... Feeling alone in these type of problems is a bit demotivating.
(Tested with UDK November 2012, same behaviors observed with UDK February 2013)
Settings used when importing my tets mesh :
-"Explicit Normals" enabled
-"Import tangents" disabled
List of FBX files exported from :
-Maya 2011 [FBX 2011.3 / FBX 7.1.0] -> Not working
-Maya 2013 [FBX 2013.3 / FBX 7.3.0] -> Not working (binormals/tangents broken : Example )
-3DsMax 2013 [FBX 2013.3 / FBX 7.3.0] -> Not working
Blender 2.63 [FBX 6.1.0] -> Working
Modo 701 [FBX 6.1.0] -> Working
So, I guess I will have to wrote a batch converter from Maya to Blender FBX files.
Unfortunately it breaks all the normals on my meshes, I get a faceted mesh.
So while handplane works, UDK is unable to import my mesh correctly.
[EDIT] Ha funny, if I use the FBX generated by Blender and import it in UDK will all the settings above disabled it import my mesh correctly.
So it's definitively related to an FBX version problem.
http://www.froyok.fr/blog/2013-04-fbx-exporter-for-udk-static-meshes-maya-to-blender-fix
It fix the vertices count problem :
Edit: Maya 2014 vs Modo 701
http://www.polycount.com/forum/showthread.php?t=124191
Was this issue resolved with the new FBX export or why didn't I ever have this problem? When I get a shading error it is usually due to wrong smoothing groups, but your problems looks like the tangents are read completely wrong, ignoring the information from the normal map mostly. I don't understand how this can work for me, even though I didn't revert to older FBX or other tweaks. Any ideas why?
FBX EXPORT:
smoothing groups: on
tangents and binormals: on
referenced containers content: on
all other options: off
UDK FBX IMPORT:
override name: on
explicit normals: on
remove degenerates: on
one convex hull per UCX: on (not necessary unless you have collision)
Important! I keep "import tangents" off -- it screws up normal map mirroring, even tho epic claims it does not.
UDK TEXTURE IMPORT
compression settings: TC_Normalmap
all other settings: default
I inverted the green channel of my normal map inside my material (in UDK) by multiplying against a 1, -1, 1 vector.
The normal map perfectly works in xnormal preview and in UDK static mesh editor (with the dynamic light):
When I generate the lightmap (the lightmap UV channel is generated inside the UDK) some of the surfaces become concave a little:
My question is why, and how to avoid this?
BTW this model is done in XSI. When I thought maybe the XSI fbx export is bad I started try exporting from maya. When I leave the normals locked, nothing is changed, (the normals ok under the dynamic light, and strange with lightmap) but in UDK the vertex number significantly higher. If I unlock the normals, and redo the soft and hard edge setup then this number about the same, but in this case I have to redo all the normal map baking because of the different tangents... How could I keep the imported normals, and keep the vertex number low when I export this XSI model from maya? (Because probably it'll be animated in maya...)
I little time ago I found the origin of the problem. I forgot to write the answer of the problem here. It my fault in the end and how I manage my vertex normal in Maya.
So what is the problem ? First I will explain how UDK import FBX : when UDK import any FBX it tries to find any smoothing group. With Maya there is no smoothing groups, but the FBX exporter is in charge to make them. Smoothing group while defined by Faces in any app supporting this system are stored per vertex from what I know.
In Maya there is multiple way to manipulate normals, the most commons are :
or
The visual result in Maya is the same, but behind the scene it's not the case. While the Soften/Harden actually work correctly and merge similar vertex normals, the Average/Set to Face orient the normals BUT don't merge them if there are similar.
So if you make a faceted mesh (every face has its own shading) and then an "Average normal" on top, you get visually a smoothed and uniform shading, but every vertex normals are still splitted, even after the export.
Here is a little test with two cubes :
In your case I think the problem come from the Tangent, try to import your mesh with "Explicit normals" enabled and "Import tangents" disabled. Also make a proper lightmap, this should help as automatic UVs can make seams where you don't want to, they could break the shading after the bake.
Anyway, see my answer above about the soft/hard normals.
From maya it's only works as you wrote (explicit normal:on,import tangents:off).
For a fbx which came from XSI the import tangent has to be set ON.
I'll export it with a 2nd UV channel for the lightmap and hope this will solve this concave shading problem. Still don't understand why it's became like that even it's perfect in xnormal and UDK model preview with dynamic light...
Its just insane how much trouble you can have just because plugins and formats and programs are not working as they should and everyone doing their own thing and nothing is normed
So is it still the case that the FBX exporter is broken as of Maya 2013, or by doing that smoothing trick should it correct the Tangents? Otherwise I have no clue what is going on...
For example, setting your entire mesh to 1 smooth/hardness group and backing out a map (with all the crazy gradient shifts) would results (if the engine is not synched) in pretty crappy looks models with the normal map, warping lights, etc.
A synched engine with the baker (which takes the proper tangent and vertex information from the source) will result in MUCH better results, and usually for the cheaper, since in some cases you wouldn't spend all day creating new uv's, cutting seams, smooth grouping, etc every nook and cranny to avoid issues.
You should read EQ's sticky in Technical, there are the A to Z's about all this stuff and tangents.
http://www.gamedev.net/topic/609108-normal-mapping-tangents-binormals/
That's why it's currently a problem as Ace-Angel point out : the way we pre-compute the tangent/binormals change for one application ton one another. Maya doesn't compute them as UDK for example.