Maya alpha channel mask for compositing?

malcolm · Dec 2009

So I tick the box, select production quality from the drop down and render hoping to get a mask out so I can composite a skybox in later. The mask is not accurate and there is a 1-2 pixel outline around everything when I use the alpha channel as a mask. What I'm I doing wrong I thought this would work.

bugo · Dec 2009

Are you using MentalRay or Maya Software?
In Mentalray there's a premultiply function that if you turn it on will help you out selecting the mask in photoshop without getting any inaccurate background colors from your render. Also you would like to mess around with the material Matte Opacity? Or even creating a background material to hold all the background so you can only grab what you want above that material.

bejkon · Dec 2009

Think it's its using the alpha fot aa purposes.

malcolm · Dec 2009

The problem is not setting up the mask the problem is the mask is not the same as the render. It's as if the antialiasing is off by 1 pixel. I'm using mental ray maya 2008.

malcolm · Dec 2009

I don't have any alpha on the models, basically the problem occurs with a cube on a black background. There must be a way to get this working or you wouldn't be able to composite anything at all in maya?

malcolm · Dec 2009

I can't use a surface shader because the skybox would block the procedural environment sky I'm using for the final gather/global illumination.

commander_keen · Dec 2009

You are aware that you have to un-premultiply anything with AA when compositing?

malcolm · Dec 2009

commander_keen, thanks. Yeah a guy at work also had the answer I had to turn off premultiple and turn on alpha clip in the frame buffer options.

commander_keen · Dec 2009

Yeah generally this is done in the compositing app though. You can "undo" the blending with the background that happens in most renders. I guess you could call it Post Divide. I think the math would be (ImageColor.rgb /= ImageColor.a) assuming the background color is black. Most compositing software allows you to import images as "premultiplyed" which will do this automatically.

Maya alpha channel mask for compositing?

Replies