Normal issues exporting from maya and importing to UDK

Jbird

Hi, I think other Polycounters has been circling around this issue but I haven't found anyone that really has nailed the issues I'm having.

I've noticed that when export my meshes from Maya and import my models to UDK the Normal map doesn't display correctly, I have devoted some time to find out where the problem lies. I've made a pretty simple test mesh to trying exporting and importing with different settings.

The mesh displayed in the pics below has all hard edges except the cylindrical objects, they are smoothed. Also pretty much all the faces are on their own seperate UV-Island with a padding of 10 units.

As you can see it looks good in Marmoset, but in UDK the mesh looks pretty harsh and not very smooth. I have exported it as a FBX 2012 file and I have also check the "smoothing group" box in the FBX prompt window. And when I import to UDK I have checked the box for "Explicit Normals".

The thing is that if I export from Max instead of Maya using Smoothing groups my model looks pretty much exactly as it appears in Marmoset.

My big questing is: Is there anyway to get the same result when exporting from Maya as I get when exporting from Max using Smoothing groups?



Maya



Marmoset



UDK

Jbird

The sharp edges are a little more apparent in this pic.

Cheers!

passerby

could be 2 things, for UDK you need the green channel inverted udk uses +x -y +z, and the tanget space between the baker and udk is different

Jbird

It is set to -y, so that's not it. :S

passerby

the tangent space than.

this will be impossible to match completely but when exporting out of maya to udk, use fbx and check off export tangents and bi-normals.

and import it into udk with explicit normals checked.

Jbird

looks the same, I have put the maya and max models next to each other in UDK now, and they look pretty much as bad, maybe I'm just picky, but i thought I would be able to get better results in UDK fo'sho.

DennisGlowacki

Sounds like you did everything right I'm surprised it has that hard edge hm.

You could try an opposite approach instead of having the edges hard with uv seams, weld your problem edge and make it a soft edge then rebake.

Jbird

hmm, yeah i'll might try that out, I'm gonna research some more and see how everybody else achieves a good look.

Jbird

I think I'm gonna try and make a bake when the model geometry isn't all on separate UV-Island and instead keep bigger combined Island, I've noticed that a lot of the models in the Unreal maps use that set up.

But what do you guys do to get good results with your normal maps in UDK? do you unwrap the mesh into single UV-islands or do you keep bigger island? do you smooth or not smooth?

Ace-Angel

You can't get good results because UDK doesn't have a synched baker for the Normal Maps currently for the tangents, which is the math so you get 'correct' looking Normals for the specific viewports.

If you want 'nice' normal maps, what you have now is fine, add in Specular and Diffuse and the issue should be extremely minor. If you want perfection, then go back to the older versions of Max and bake from there, although if going back to a broken and slower baking system of Max to compensate for something else that is broken isn't exactly ideal, so your mileage may vary.

Jbird

thanks for the reply Ace-Angel, yeah from what my experimenting tells me your most likely right. Adding diffuse, spec with details in them will most likely get rid of the more apparent edges seen now with only normal plugged in. Adding different lighting and post process things will probably further less apparent.

My latest experiment with big UV-chunks and pretty much only one smoothing group for the mesh didn't give a very good result in UDK, worse than the previous version. This is something I've seen before on ze interwebs, the ugly shading you get for having one smoothing group and big combined UV-island instead of seperate ones.

I'm gonna texture some of the models of my main scene now and see how that can improve things. I've read quite bit about this on the internet and I've seen a bunch of pictures describing the issues I've looked in to here. But It is very educating to try out all different setups and see what impact it has.


Thanks for the feedback and info guys.

basically one smoothing group and welded UV-islands = bad result, weird shading

DennisGlowacki

Did you redo the bake on the single smoothing grouped mesh because it shouldn't look that bad ;O hmm

Jbird

hmm okey, I feel you. But this is the results I got:

In the pic below we have from left the smoothed low-poly mesh with stitched UV-islands and to the right the hard edge low-poly exported from max. The not so nice looking shading of the smoothed-version is quite apparent, although I like some edges, the light just looks weird on it. And yeah, I made a few tweaks on the smoothed version, I hardened the end cap of the pipes and inverted the Y-axis of the normal map since I forgot it the first time around.




Normal map of the smoothed mesh, with stitched UV-Islands.



All hard edges and exported with 3DSMax using a lot of different smoothing groups

Ace-Angel

DennisGlowacki wrote: »

Did you redo the bake on the single smoothing grouped mesh because it shouldn't look that bad ;O hmm

Maybe if you use Support-Loops, but by itself, no, they will look this bad in UDK.

Harbinger

Don't make your normal maps work harder than they need to. Each side of that four sided object should be UV mapped as it's own UV shell and the boundaries of that shell should have hard edges.

Seeing a lot of gradient and color variation over a surface that should render as flat is called "stressing" your normal map. Because you baked that surface with averaged (or "soft" in Maya terms) normals, to the renderer it's basically a rounded surface. Your normal map then has to compensate and work really hard to fix that surface rounding to make it render flat again.

The solution is to harden the edges. However, when you harden edges you need to separate those edges in the UV map to allow some pixel bleeding, otherwise you'll end up with seams due to texture compression and/or MIP mapping.

King Mango

I have hell with normal maps in Maya. But the best results I've gotten are from making the target object all soft, and the source all hard. I just throw in enough polys on bevels etc so there are no crinkles.

If I don't do that I find that what I get is just the opposite of your problem, overly rounded faces. Like I have a two unit bevel on a corner but the normal tries to make the flat face half-round anyway. Aggravating as hell. And I have followed a shit-ton of tutes to no improvement.