I'm in the process of texturing a creature in Maya 2011 and I'm encountering an issue with alpha mapping that has me stumped. I don't know the program very well to begin with, and I've done a fair amount of research on this particular problem, but it has yet to be resolved and has become a frustrating roadblock for me.
Basically, I can't get my alpha / opacity maps to look right... Mental Ray in particular makes the entire model look really wonky.
First thing I did was save my color map as a 32 bit .tga with the alpha maps included. In Maya I added a new Blinn texture to the model, added the color map and rendered it out using the default Maya Software settings. What I got was this:
First of all I would like to know, what's up with that "ghostly" image on the alpha mapped polygons, and how do I get rid of it? I understand it's just lighting that is reflecting off of the polys (I didn't have this problem when I used a lambert shader), but how do I make it so that the black part of the alpha map doesn't pick up any lighting info?
I switched to Mental Ray and I then got this:
Now the colors just look dull and the alpha lighting issue is even more apparent.
I did some searching on Google to try and find a solution, and one of the tricks that is supposed to work is to go to the Color Balance menu in the texture file and check "Alpha is Luminous". So I did that and tried rendering in Mental Ray again:
Yeeeeeah that's not it.
There was one other thing I found, and that was to make sure that "Pass Custom Alpha Channel" was checked in the Custom Entities menu inside the Render Settings (which it is). The only other thing I could think of was breaking my alpha and my color textures into two separate 24 bit files and throwing the alpha in there as a transparancy, but that didn't help much either:
It got rid of that ghosting effect but clearly the alphas aren't even being recognized by Mental Ray, and the character's color map still looks desaturated. I would really appreciate some help with this issue... I am totally stumped and I feel like I'm just wasting time sitting here and messing around with a bunch of settings in hopes that SOMEthing will work. =\
Replies
The "mental ray Arch&Design material" got a "Cutout" map-slot, put your image there and click that "alpha as source" (or whatever it's called).
And for the normalmap you could then use that "normalbump" material in the bumpmap-skot.
But why would you use MR for a lowpoly model anyways?