Vertex color question for maya

tehrobster2

So I have been reading up on Vertex color, because I need to use it for the blend layer inside cryengine 3. So from what I read about the blend layer I will need a vertex alpha channel and a blend map. And what I can gather from the crydev forums that the blend map is just an alpha map. So is the vertex alpha channel something that you save into the mesh it self through vertex color ?

CheeseOnToast

Yes, it is. To create a new vertex colour set in Maya go to color > color set editor. With the object selected, hit "new" and rename it to whatever the exporter requires it to be called ( it may not need a specific name though, in which case just call it whatever you want ). You should now be able to paint vertex colour/alpha with either "apply color" for flood fills or the paint vertex color tool.

tehrobster2

Okay awesome! Thanks so much, and to get the alpha from the vertex color that I painted I just do a batch bake ? And again this really helpful.

tehrobster2

Okay so I got it to bake to the mesh, now I am just trying to get it produce the alpha map, but the one its been producing are black, any help would be great! here is what I baked the vertices with.

kodde

I'm not following what you are trying to produce here. Are you trying to transfer your vertex colors to a texture? Then what's the point of vertex painting to begin with? I mean with your polygon density (based on that screenshot) you can probably get higher fidelity by just going straight to working with that texture.

If you want to save your vertex colors to texture then open the options for the Paint Vertex Color Tool, look under Attribute Maps and Export. Should be straight forward from there.

tehrobster2

I apologize for being confusing, I am trying to use the blend inside Cryengine 3, and so it says I need a vertex alpha channel and a generic blend map, so maybe I confused the alpha map for the generic blend map, here is the page I was reading http://freesdk.crydev.net/display/SDKDOC2/Blend+Layer and again I apologize for any confusion. I am still learning vertex color and this engine.

kodde

I see.

It's just a matter of painting the vertex color alpha information and exporting the mesh. The generic "blend map" is not nessecarily created in Maya, more than often I'm guessing you create this in Photoshop using a heightmap as a base.

There shouldn't really be a need for you to "bake" or transfer the vertex colors to texture from what I can tell.

I've played around with doing similar things with my own shaders, although instead of using vertex colors for defining placement of material A & B, I've used a texture.

kodde

Are you using an export format that allows you to export vertex colors? Are you sure the vertex colors are being imported properly in your engine?

passerby

export out vertex coloured mesh, and the blendmap is either made with a heightmap or by hand.

all the blendmap does is tell the engine what areas to blender the vertex paint into first.

tehrobster2

Ohhhhhhhhh! Thanks guys this makes way more sense! This has been a ton of help, I will post my results soon! And again thank you!

tehrobster2

So I finally got it working thanks to you guys, and I am going to write what I had to do for anybody in the future, who tries to do this with maya. So I had to use the color set editor like Cheese on toast said. But first I have to add a color set just RGB, because the alpha setting will not let you use the vertex paint tool. So paint what you want to have subtracted in your blend mode, and then once your finished you have to go back to your color set editor and then add the alpha set, and after you do that click both of them, and hit the blend button, and it will blend the two sets together, so now you can export using the cryengine export tools and make sure you have export vertex colors clicked.

juggulator

Could you possibly detail the process step by step? Im confused as to what colors sets to use. Is it RBG + RGBA or RGB + Alpha set? I used RGB + Alpha (I think) but can't get it to work. The blend layer shows up but not where I paint and enabling Vertex Colors in CryEngine only shows black as the examples I've seen show painting black to make the blend layer fully show, white for no blend showing and gray for in between. CryTek doesn't give much detail as to how it's done in Maya. It sounds like your painting on the Alpha set but does any painting need to be done on the RGB layer, are you using Add or Replace? Both operations paint alpha on the mesh. Any help is much appreciated.

e-freak

i'm not sure which build you are on, but iirc the freesdk supports a blendmap just as well as vertex colors for the blend shader. Might be a lot easier to use.

Thanks for following up with your solution though on the color-set editor - i know a few people who ran into similar problems with it!

devingeesr

Thanks for the info CheeseOnToast it helped out a lot.

NakedColors

My vertex is weird. I don't know how or what it's called but how do I make it to the default color (Yellow with blue)?

Picture: http://imgur.com/shPkefc