Maya Normal Maps Flipping Green?

Tulkamir

Hey, I've got a problem where when I bake normals in Maya my green (vertical) channel seems to be flipped. I'm not sure if this is a Maya problem, something I'm doing wrong or what. One thought is that it could have something to do with Maya using a different up axis than direct x programs (I've only test rendered the map in CrazyBump and our engine).

Anyways, basically I'm wondering if this is something I'm doing wrong, a known problem with Maya, something else, or if I'm just a bit nutty.

Thanks a bunch.

[edit]Oh, and if I am right about this problem it should fix it to just inverse the green channel in photoshop, right?[/edit]

Daz

Yeah you answered your own question Mitch. Unfortunately there is no standardization, and what Autodesk thinks is a correct normal map not everyone agrees with. Maya spits out normal maps that are correct for me in its viewport of course, but 'wrong' in our engine. It's a royal pain in the arse, particularly If Im mixing normals in Crazybump that are generated from different places, and neglected to stay on top of wether or not their green channels are consistent.

MoP

Yeah, inverting the Green channel should fix it. Probably want to re-normalise after that just in case.
I'd be very surprised if Maya didn't have an option for choosing which direction to bake each channel, since every other baking app that I'm aware of does. It's probably just well hidden, or you haven't looked for it hard enough

Tulkamir

Alright, thanks a bunch guys. It's not a huge deal to fix I supose, but like you said Darren, it's a pain. I did look around for the option you're talkin about MoP, but didn't find anything. I'm gonna look a bit harder, but if anyone else knows of such an option in Maya I'd love to hear.

Tulkamir

Hmm, alright, so after looking into it it seems that there is a way to fix it. Before baking the normal maps go into the object's shape attribute editor and under tangent space switch it from Right Handed to Left Handed. I haven't checked it out fully, but it seems to do the trick. Although I can't find a way to automatically make it so all objects are created with Left Handed Tanget Space, and I don't know whether or not it will affect how the meshes export to the engine yet.

Illusions

Window > Settings/Preferences > Preferences > Settings: Modeling > "Polygon Tangent Space" : Default Coordinate System (Options: Right Handed 'Default' / Left Handed)

What version of Maya are you using? Because Transfer maps seems to generate normal maps properly for me (at least they display properly in both xNormal and Mental Ray, don't know about any game engines).

Daz

Hey so is that in 8.5 Illusions? I dont see that in 8.0...

Illusions

Yeah, apparently its new in 8.5...just checked a copy of 8.0, and nope...not there.

Tulkamir

Daz, the same setting is under the object's shape node attributes. It's not ideal, but I figure a that a quick mel script should make it manageable.