Maya hardware transparency problem

sagiarts · Feb 2010

Hi I am facing a wired problem of hardware display (high quality) for transparency maps.
However I keep the map as a alpha channel or as a separate tga file they give the same result like this:

Please help me guys

Hazardous · Feb 2010

Viewport opacity in maya ROFL ? If someone DOES find a remedy to this, PLEASE POST IT!!!!!

vofff · Feb 2010

I think in Maya view port cant handle the transparency. Try uncheck the poly transparency in shading and check the att editor render if their are single side if so uncheck it and click on double side.

AreDub3D · Feb 2010

That's definitely one of the funnier examples I've seen of it ...

But sorry, that is actually not a "weird problem" as far as Autodesk is concerned.

Unless a user has taken it upon themselves to make a fix (and I'm not aware any have), you'll need to fully render it or just go ahead and drop your model in to a game engine to see what you've made (if you're using UDK, at least, it's easy to update the changes you make in Maya or Photoshop.)

This is one of the great Mysteries of Maya ...

(That's some nice forehead skin, by the way, Sagiarts! And welcome to polycount ... helpful folks here. Well, maybe not me, but we try.)

MoP · Feb 2010

Maybe try finding a CGFX or HLSL shader that supports the maps you need, they tend to look better than the standard Hardware Display shader.

chronic · Feb 2010

the fix for this is fairly simple, although annoying, you need to make 2 shaders - one for objects that actually have transparency and one for objects that dont. Even though you objects may be sharing the same textures and alpha channel you need to disconnect the transparency option in the shader for those that don't actually have transparent bits.

hlsl and cgfx shaders have their own inherent problems with transparency in maya - and i dont recommend them for this purpose. they can be made to work under limited circumstances but multiple transparent layered objects will never work properly (like hair).

ivars · Feb 2010

This is no way weird, magic or a mystery. It's alpha-sorting. It's an issue in all realtime rendering. Maya viewport uses object transparency sorting, based on the bounding sphere I think.
Chronic gave you the best solution, but most engines would prefer a limited use of alpha, using only alpha where it's needed (hair) and not use it for the whole mesh. ie. two separate textures.

AreDub3D · Feb 2010

ivars wrote: »

This is no way weird, magic or a mystery.

The mystery I was referring to is why it hasn't been made to work out of the box in the last three versions I've been using Maya.

The days of expecting your more enterprising customers to fix your software *should* be over. (Not ranting, here -- I love Maya. Just calling a spade a spade!)

kodde · Feb 2010

I'll second Chronic's solution, a bit of a hassle but it works.

chronic wrote: »

hlsl and cgfx shaders have their own inherent problems with transparency in maya - and i dont recommend them for this purpose. they can be made to work under limited circumstances but multiple transparent layered objects will never work properly (like hair).

I actually had mutliple objects layered in front of each other with transparency using a CGFX shader. Worked fine. They had to be separated objects to work though, I think. They were all completely flat planes. Probably wont work well if its a semi-cylindrical plane and you were to see both front side and back side.

Hazardous · Feb 2010

chronic wrote: »

the fix for this is fairly simple, although annoying, you need to make 2 shaders - one for objects that actually have transparency and one for objects that dont. Even though you objects may be sharing the same textures and alpha channel you need to disconnect the transparency option in the shader for those that don't actually have transparent bits.

hlsl and cgfx shaders have their own inherent problems with transparency in maya - and i dont recommend them for this purpose. they can be made to work under limited circumstances but multiple transparent layered objects will never work properly (like hair).

Awesome dude this works like a charm, althought tedious and annoying, it solves the problem completely.

Ive hooked up 2 blinns, one with opacity and one without, applied the two to their respective pieces and everythings fine. I even tested it on a character im working on with layers and layers of transparency. perfect.

I should have remembered that from doing pretty much the same thing to get a good sort going inside marmoset!

oglu · Feb 2010

try to turn off object transparency sorting and turn on polygon transparency sorting...
in the viewport under shading...

claydough · Feb 2010

Not Sure if this env still works fer viewport alpha.
But jes in case:

MAYA_SLOW_DRAWPIXELS = 1

chronic · Feb 2010

was reading from the maya docs about cgfx:

Maya supports rendering of transparent CgFX shaders. Maya detects whether the blending state is enabled in the first pass. If blending is enabled, Maya uses the Object Transparency Sorting or Polygon Transparency Sorting options in the viewport to correctly render the transparent shader.

So maybe cgfx is our salvation after all - HLSL for sure is borked

Hazardous · Feb 2010

oglu wrote: »

try to turn off object transparency sorting and turn on polygon transparency sorting...
in the viewport under shading...

Thanks for pointing this one out, it works a charm for non normal mapped stuff. But anything with normals + opacity still end up looking pretty funky and random.

kodde · Feb 2010

Maya 2009, using CGFX shader

Hazardous · Feb 2010

Kodde: Cool!! But try an example, thats normal mapped, all one object and has parts with and without opacity, as THAT is the main culprit thats breaking things.

The example you have there works perfectly even now in maya 8.5, with the standard blinn shader.

kodde · Feb 2010

I don't think it will work when it's all one object, same issue as regular Maya. My point with example 2 and 3 in my image was that it sorts them correctly (seems to?) even though one object's pivot might be closer to camera than the other.

EDIT: I can give your suggestion a try though, will do it later in the evening when I'm at home.

chronic · Feb 2010

the most annoying transparency problem I have seen is like this example from Xoliul's shader thread:
http://boards.polycount.net/showpost.php?p=1075588&postcount=195
this happens with HLSL shaders and I was under the impression it happened in cgfx too, but if not then I will be fully converting to CGFX.

sagiarts · Feb 2010

Thanx a ton man(chronic,kodde, Hazardous,claydough,oglu,vofff....and the list of helpful people continues) thanx alot,I really mean it..... cause nobody even bothered to respond in sites like zbrushcentral or cgtalkand I get sooooooo many helpful responses out here.
I request Admin to kindly keep an auto mailing system to the users mailbox with each response to his subscribed thread(it is not there in my case so I took a while to write this mail)
I will be experimenting with all this so valuable responses and am sure I will come up with a solution shortly.
Pals I also need to know if there is any training dvd on game level modeling texturing in 3ds max,its damn urgent.
Thanx again to all my kindhearted freinds who spent their valuable times responding this thread.
Kind regards
Arnab
AKA Sagiarts

kodde · Feb 2010

I am having quite similar problems with CGFX shaders, not that convinced any more.

chronic · Feb 2010

@Kodde
Yep, i just finnished converting one of my HLSL shaders to CGFX and was disappointed as well - single or double layers of transparency could be corrected with creative object combining - but it seems like complex multi-layered transparency will always be a problem.

My cynical view is that this is one of those minor fringe case issues that Autodesk will never get around to resolving.

fantasymaster · Feb 2010

Had the same problem with the alpha chanel. I used separate alpha textures to fix it.

Hazardous · Feb 2010

chronic wrote: »

@Kodde
Yep, i just finnished converting one of my HLSL shaders to CGFX and was disappointed as well - single or double layers of transparency could be corrected with creative object combining - but it seems like complex multi-layered transparency will always be a problem.

My cynical view is that this is one of those minor fringe case issues that Autodesk will never get around to resolving.

Its such a shame, but ive got to agree with you man, I dont think they will get around to it either.

Alpha sorting has been a bane to game development since forever. Especially in hair. Metal Gear Solid/ Final Fantasy are one of the few places that dare actually give characters hair styles that arent crew cut / short cropped non opacity styles, I dont ever recall seeing alpha sorting problems in Final Fantasy.

I blame the coders. :P :P :P

bugo · Feb 2010

there is NO fix for hardware shading unless you use a cgfx or hlsl shader.

Eric Chadwick · Feb 2010

Vertex order usually solves it for me.
http://wiki.polycount.net/Opacity_Map#SortingProblems

chronic · Feb 2010

I've decided to say "screw you alpha blending" - our engine here at work does alpha testing anyways so why bother with other techniques.
replace your blend code with alpha test code - seems to work in preliminary tests across all situations and isnt influenced by mayas viewport Shading>Object Transparency Sorting or Shading>Polygon Transparency Sorting
CGFX code:

//BlendEnable = true;
//BlendFunc = float2(srcalpha, oneminussrcalpha);
AlphaTestEnable = true;
AlphaFunc = float2(Greater, .5);

Eric Chadwick · Feb 2010

Yeah alpha test solves the sorting problem, but it's only on/off, no translucency. Can't use it for hair, for example.

You can use both together at the same time. But it only reduces the sorting problems to a fringe of pixels, not really that helpful.

coldkodiak · Feb 2010

Ahh, the age old problem of alpha sorting...

/me sips his bourbon

when will we learn?

Hazardous · Feb 2010

coldkodiak wrote: »

Ahh, the age old problem of alpha sorting...

/me sips his bourbon

when will we learn?

When will YOU learn how to fix it and be the awesomest person on the planet!!!!

chronic · Feb 2010

One day a hero will rise and say:

Fabulous secret powers were revealed to me the day I held aloft my magic sword and said... By the power of Greyskull! I have solved alpha transparency sorting!

until that day . . .

Maya hardware transparency problem

Replies