UDK problem with normal maps

Lightweight

I don't know if this is the right forum but then again, I didn't find any other forum that was particularly more suited for this question.


I'm very new to working with game models and textures and I just started using UDK. I'm building a jeep @ 20k polys incl interior, and when Im importing the mesh into UDK and applying the normalmap for the hood, the results are horrible.

I made a picture Im currently spreading around some forums so people can get an idea of what my problem is:



Fullscreen @ http://www.cambranddesigns.com/public/UDK_problem.jpg

These highlighted edges look really smooth and fine in Maya:



Fullscreen @ http://www.cambranddesigns.com/public/Maya_UDK_smooth.jpg

When I import the normal map into UDK, I make sure to set it to "TC_normalmap"-compression, which is what most people do. I tried both .tga and .png

Can anyone shed some light at this and what Im doing wrong?

Since Im just testing right now, the car itself is exported as a combined, triangulated mesh.

Regards

Ace-Angel

When importing the model, in the latest version of UDK, there should a Pop-Up Box that asks something about smooth groups set to average (don't recall exactly) or one. It could be that, or export with one smoothing group set to Auto. (Sorry, don't know what the term is called in Maya, but in Max, it's called Smoothing group).

Also, how did you bake the normal map? I recall that UDK needs a flipped Y Channel, kinda like Max, unless they changed that, so it might be worth a look.

passerby

when importing the model tick off use explicit normals.

Lightweight

Hey thanks for the replies dudes!

Regarding the importing of my .ASE file (using Axmesh command in Maya to export) I get these options only:



Could this be because it's ASE and not some other format?


Regarding "Smooth Groups", Maya don't use those at all, but only using soft/hard edges, and hard edges are supposed to get split/separated in the UV. Like this:



The normal map was generated using Mayas Transfer Maps. I will attach the normal map so you can see it:





Regards

passerby

ah sorry iv didn't know the option i mentioned was only for fbx.
you could try fbx with that option though.

thinking using fbx with explicit normals should work udk will still give you a error about smoothing groups but it all looks perfect in engine.

i could test things out for you tonight since i just got maya 2012

percydaman

This might have something to do with how maya creates normal maps. I think they need the green channel flipped. Have you tried that? Might try creating the normal maps in another package like xnormal or 3dsmax. I can help you out with 3dsmax if you need a hand.

Lightweight

passerby wrote: »

ah sorry iv didn't know the option i mentioned was only for fbx.
you could try fbx with that option though.

thinking using fbx with explicit normals should work udk will still give you a error about smoothing groups but it all looks perfect in engine.

i could test things out for you tonight since i just got maya 2012

That would be very kind of you! I've been using that Ax-plugin or what its called and exporting the mesh to .ase. If exporting to FBX, I could probably do that with Maya itself and no plugin I guess. I will try the .fbx when I get the chance, and Im eagerly waiting for whatever you might find out with 2012

passerby

can you link me the ASE plugin i cant find one that works with 2011 or 2012.

but i did some testing with the fbx and as long as i check off explict normals it workd flawless and i get the exact same results in udk as what i have in maya.


is there actually any function you need that isnt in fbx?

Lightweight

passerby wrote: »

can you link me the ASE plugin i cant find one that works with 2011 or 2012.

but i did some testing with the fbx and as long as i check off explict normals it workd flawless and i get the exact same results in udk as what i have in maya.


is there actually any function you need that isnt in fbx?

http://udn.epicgames.com/Two/ActorX.html

I will also attach the .obj for the hi-res and low-res in case you wanted to inspect them further:

www.cambranddesigns.com/dnd/hires_hood.obj
www.cambranddesigns.com/dnd/lowres_hood.obj

Regarding filetypes and possible needs, I don't really have any. I will import this simply for creating a scenery with the car in it eventually to go into my portfolio. I haven't tried importing a normal map for a plane surface that would define i.e. asphalt or anything like that. The only thing I've tried is these normal maps I get from baking hires -> lowres, and so far it pretty much wont work for me. I tried flipping the green channel on some of my maps, and in some places on the mesh, it gets less horrible, but still far from OK, so I don't think thats the problem

Regards

TorQue[MoD]

I think Passerby is right, the normals for UDK need to be created with the Y inverted if you're exporting from Maya. Try using xnormal (http://www.xnormal.net/1.aspx) to re-generate the normal and make sure you change it to UDK under the Bake settings and see if that helps.

Also, the UDK forums are pretty helpful as well...http://forums.epicgames.com/forumdisplay.php?f=366

passerby

ya im still trying to figure it out it is with the normal map i eneded up smoothing the whole mesh and bring it in udk and it gets the hard edges once you put the normal map on it.


thinking it is a sync thing between what your using to bake and udk.

i recreated the normal map useing the HP and LP you provided which looks pretty perfect when viewed from in xnormal.



that is the LP with the normal map applied in the xnormal viewer.
but when viewed from udk i get the same issue and i already baked with green channel inverted.


EDIT:
after i bit of messing around it seems the engines makes all UV Seams hard so it seems you need to rework your UV seams and layout.

percydaman

yeah I'm getting the same thing. Looks fine in 3dsmax or maya viewport, but in UDK, the normal map is ignored along uv seams. This is something Ive noticed before and commented on the UDK forums, but never got a good response. Doesn't seem like something that should happen.

Ace-Angel

Did you create a second set of UV maps? Although it shouldn't be the case, it might help (for channel 2).

TorQue[MoD]

Second set of UV maps is only for lighting.

Ace-Angel

Yes, as well for symmetry, I know that, but considering from what I read in the thread, he pretty much tried everything.

His normals are fine, he must have flipped the Y channel by now, and his UV's are not symmetry based and he for sure must have ticked and unticked every single box and (who knows) even tried an FBX export.

He doesn't need a second channel, but at this point, while it will not help, he might as well put in the checklist, who knows.

Lightweight

This is confusing for sure. Are there any other freeware engines I could try this in, and see if it's just UDK being bad or if this goes for pretty much any game engine?
My friend works @ Dice and he's working on Battlefield 3 right now, and it seems that he's able to do what I do and still get the job done, but not UDK.

I've tried flipping the green channel as well, and also creating a second channel. No go

Stromberg90

Well you could try the trial version of toolbag or unity?

m4dcow

I got it to work okay in UDK besides a few weird lighting issues, the normals looked okay.

UDK on the left and Maya on the right.

TorQue[MoD]

Like I said, try generating the normals inside another program like xnormal. This way you'll be able to figure out if the issue was due to the normal or the mesh itself.

Lightweight

m4dcow wrote: »

I got it to work okay in UDK besides a few weird lighting issues, the normals looked okay.

UDK on the left and Maya on the right.

How did you do it?

TorQue[MoD]

I tried generating them in xnormal, both with +Y and -Y but doesn't change anything Im afraid. Also, the normal maps I got from xNormal were quite low quality, like there was noise in the maps where there definitely shouldn't be any. But thats another topic cause Im sure I could tweak the values and get a better result.

I followed this tutorial from start to end and my results were horrible:

hxxp://www.corcoran3d.com/tutorial_videos/xNormal_Intro.htm

cryrid

I will try the .fbx when I get the chance

Have you?

passerby

cryrid wrote: »

Have you?

i have the olny way i was able to get rid of the hard edges was to redo the UV's

m4dcow

I exported the low poly from maya as fbx and imported into udk with explicit normals.

Lightweight

cryrid wrote: »

Have you?

I did try it but no luck. I've been testing so much back and forth with like 10 different exported models and normal maps so I will do what m4dcow did, and see if that helps me. I will post here when I have.

Regards

TorQue[MoD]

If xnormal is exporting noise etc. its likely a problem with the mesh itself and not the normal generation of maya or xnormal. The only other thing I can think of is maybe you missed welding some verts together? I use max not maya but I know this has happened to me occasionally in max.

I can't remember if you tried exporting a new mesh to see if the same problem is happening with it. Try something simple with similar edges like a chamfered cube.

percydaman

m4dcow wrote: »

I got it to work okay in UDK besides a few weird lighting issues, the normals looked okay.

UDK on the left and Maya on the right.

Is this screenshot before or after building lighting? I've noticed that the issue only pops up for me most of the time after building lighting using lighmass. Until then, the engine treats it as a realtime shaded/shadowed object.

Lightweight

The strange thing is that the model + normal map works fine in Maya, Max and Marmoset Toolbag. Since this is only a demonstration I think I will settle with presenting it using Toolbag instead of UDK. The possible companies that I know that I'd work for don't use UDK either way, and this is taking up too much time I'm afraid. It's a shame though, since learning UDK would look good on ones resum

TorQue[MoD]

Yes but often times what works in a render package won't work the same in a game package. Form the look of the normal map in UDK and the way its pinching into a V shape at the front bottom there I'd definitely say there's something wrong with the mesh. Like I said, either make sure all verts are welded together or simply try re-extruding those faces. Might fix the problem.

m4dcow

@percydaman
It's after building lighting. Also to not the green channel of my normal map is flipped for UDK. I suspect the weird issue in the bottom left may be hard egdes where they should be soft, but I deleted the files already.