Lightmass wont render an OpacityMask that uses vertex color

jocose

Hey Everyone,

I have some glass that I wanted to have a layer of sand sitting on in my mission. I figured this would be a reasonable place to try and use vertex colors.

I made an evenly subdivided plane and set up a simple material that just took my sand texture and masked it using the red chan in my vertex color node (I just plugged it directly into "OpacityMask".)

This worked great, I was able to paint on the mesh and get some nice patches of sand.

Problem was that lightmass cast the entire polygons shadow rather than just the unmasked portion. Dyanmic shadows render fine, but as soon as I bake my lightmaps it creates a big block.

I know my material is set up correctly for this to work because if I use another texture with alpha (such as some of the leaf textures that come with UDK) those render shadows correctly (only the unmasked leaf casts a shadow instead of the entire polygon).

Anyone have any ideas on how to get this working, or is this just a fundamental limitation of LightMass?


EDIT: Here are some pics:

Dynamic Shadows Work on both the vertex painted plane and the simpler alpha mask plane.




As soon as I bake my lights I get this:



Here is what the other shader looks like:




Both materials have identical settings other than the node trees displayed. I just duplicated one to create the other and changed nothing else.

Both have their Blend Modes set to "BLEND_Masked"

divi

maybe (not sure on this) it's because vertex colors are tied to the vertices and the interpolation between them. because of that you can't offset vertex colors as well which made me very sad at some point . and maybe lightmass doesn't regard the vertex shader for opacity calculations. im not a programmer, so i don't really have a clue, but there are a lot of things you can't do with vertex colors and maybe that's one of them.

Ben Apuna

Maybe you could do something like those advanced terrain vertex blend materials that use vertex colors as a mask to tell where blend between two sets of textures using a height map.

See also this thread.

JordanW

Lightmass doesn't take into account certain attributes. Anything that takes place in worldspace, viewspace, or on the model. Basically Lightmass creates a tiling texture that it uses to represent the diffuse map of the material. This means any time you have world coordinates doing something crazy or vertex colors doing something crazy you need the lightmass replace node to give lightmass something "neutral".