[Maya] Blending texture based on vertex colors ?

Froyok

Hi folks !
I have already done some search before asking this but I can't figure out how to setup this type of shader inside maya.

I know there is already some topic about this on this forum but they don't explain how exactly the setup is done (it's only some screenshot of the shading network without naming the inputs/outputs).
Like : http://dev.polycount.com/forum/showthread.php?t=94077

What I would like to do is a very simple vertex blending between at least 2 texture (for example based on the Red vertex color).

I have tried to setup a simple shader with a layeredTexture node, but I don't understand how I could connect the vertex colors (I don't even know where I could find this vertex color output) of my mesh to the alpha input of one of my texture for blending it. I would like to use the vertex color because later my goal is to export the mesh to UDK (I would like to save my time by doing my vertex blending inside maya instead of doing it in UDK).


Any ideas ?

m4dcow

I have tried this a bunch, and at this point it isn't possible in real time without a custom shader. You can use the CPV node to do what you want, but that only renders in Mental Ray.

You could render out to get a preview, but it seems like actually doing this in UDK might be the better choice, unless you learn to program some shaders.

m4dcow

Just to specify, the links you posted are done with bitmaps or procedurals, not vertex colors.

Froyok

I see, so only a cgfx could be a solution.
(So of course, in this case I will save my time with doing it directly in UDK... )

I tried to use the one made by Brice (but it looks like I can't setup it correctly) : http://www.polycount.com/forum/showpost.php?p=834503&postcount=12

nullfed

In your example, you don't need to specifically connect the vertex colours to it.

- In your layered texture node, have the 2 textures. Make sure the one on the left is set to blend mode 'Over', and the one on the right is set to 'None'

- apply the layered texture to the model, create a vertex color set and fill it with white, alpha 1.0

- use the paint vertex color tool, set to RGBA, make the colour white and the alpha to 0 and paint on the vertices. The second texture will show through based on the alpha value


This is fine for viewport display if you're going to blend 2 diffuse textures but bringing normals and spec into it is a problem becasue normals only show up in High quality mode or viewport 2.0. Unfortunately, they display vertex colours and that will ruin the look of the material (if anyone knows a way of stopping those modes displaying vertex colours, i'd love to hear it)

As for getting this stuff in UDK, you can use Vertex Chameleon to swap values between different channels, but with the features of UDK, I say that it would be much easier to set up your materials there and also do your vertex painting there too. The process is similar and with it being your target, it will save you the hassle of getting maya to do what you want.

CGFX would be an option if you have to have it in the maya viewport but if you are heading for UDK anyway, it will take more time overall than if you just did it in UDK.

Froyok

nullfed wrote: »

In your example, you don't need to specifically connect the vertex colours to it.

- In your layered texture node, have the 2 textures. Make sure the one on the left is set to blend mode 'Over', and the one on the right is set to 'None'

- apply the layered texture to the model, create a vertex color set and fill it with white, alpha 1.0

- use the paint vertex color tool, set to RGBA, make the colour white and the alpha to 0 and paint on the vertices. The second texture will show through based on the alpha value

Thank you very much, I was making some settings in the wrong way, with your explanation it's work well with the alpha vertex color now. Most of my blending will be between only 2 textures, so it's ok by this way.

nullfed wrote: »

This is fine for viewport display if you're going to blend 2 diffuse textures but bringing normals and spec into it is a problem becasue normals only show up in High quality mode or viewport 2.0. Unfortunately, they display vertex colours and that will ruin the look of the material (if anyone knows a way of stopping those modes displaying vertex colours, i'd love to hear it)

Yeah I saw that, but since my final renders are done in UDK it's not a problem.

nullfed wrote: »

As for getting this stuff in UDK, you can use Vertex Chameleon to swap values between different channels, but with the features of UDK, I say that it would be much easier to set up your materials there and also do your vertex painting there too. The process is similar and with it being your target, it will save you the hassle of getting maya to do what you want.

CGFX would be an option if you have to have it in the maya viewport but if you are heading for UDK anyway, it will take more time overall than if you just did it in UDK.

Well, the less Time I spend in UDK, the better am I !
UDK is cool, but not very fast for this type of production imho. This is why I prefer to do a maximum of work inside Maya.

When I will have the time, I will probably try to setup a simple cgfx shader for doing that with multiple vertex color channels.

Lamuness

I'm going to necro this thread because it's really getting on my nerves and I've been trying to search for an answer for a week now.

I've tried all the steps in creating this layered texture in preparation for blending with vertex colors.

My material is set up properly, I have two UV sets, and everything. But when I apply the vertex color, my whole model turns white. After when I try to paint the weights for the textures to blend, they're actually painting the vertices' alpha, making the model transparent.

Is there a mode I need to turn off? It's on High Quality render for the viewport. Please help because I've been searching too long.

Froyok

Maybe it's the render of the viewport, I use the classic one in my case.

Lamuness

Ok, that is pretty dumb, it shows up now. Is there any way for this to show up in High Quality mode?

Froyok

I'm not sure if it's possible unfortunately. The High Quality render has some limitations.

Lamuness

Thanks, I guess I'll have to do it with 3d paint tool.

Lamuness

Have you figured out how to blend two normal maps together using the layered texture?

It doesn't seem to like it when I do it. I want to make a concrete wall, and paint in with vertex alpha where the brick will show up. It doesn't process the normal maps at all when I set it to a layered texture.

Froyok

I never tried it.
Can you show your network in your material ?

bugo

As I pointed in another thread here: http://www.polycount.com/forum/showthread.php?t=54839&highlight=terrain+vertex+shader

wr_uk

Has anything changed since maya 2012 with regards to how the vertex color painting works on blending textures? I'm running 2014. I have the exact setup as 'My material is set up properly, I have two UV sets, and everything. But when I apply the vertex color, my whole model turns white. After when I try to paint the weights for the textures to blend, they're actually painting the vertices' alpha, making the model transparent. '

The only difference is when I switch back to default quality rendering to see the results nothing has changed! the texture is solid and there are no masked areas.

Any ideas as to what the issue maybe?

Im actually beginning to think that its no longer possible to achieve with the workflow

Wayne

Aerashi

wr_uk wrote: »

Has anything changed since maya 2012 with regards to how the vertex color painting works on blending textures? I'm running 2014. I have the exact setup as 'My material is set up properly, I have two UV sets, and everything. But when I apply the vertex color, my whole model turns white. After when I try to paint the weights for the textures to blend, they're actually painting the vertices' alpha, making the model transparent. '

The only difference is when I switch back to default quality rendering to see the results nothing has changed! the texture is solid and there are no masked areas.

Any ideas as to what the issue maybe?

Im actually beginning to think that its no longer possible to achieve with the workflow

Wayne

I'm using 2014 and getting this exact same issue. It's driving me nuts. Anyone know how to fix this?