Home Technical Talk

Maya Bake Vertex Color, One Color Per Polygon

Is there a way to bake vertex color in maya without blending between vertices? I'd like to have each polygon have a single color, like the models from Polyworld: http://forum.unity3d.com/threads/polyworld-for-unity.239365/

Btw, I'm using Turtle to bake the vertex colors in Maya - is this the best approach? Thanks!

Replies

  • Jeff
    Hi Inanimate,



    You can easily get the result, but in two steps.

    The baking process is, as far as I know, would it be MR or Turtle, not able to set baked colour on a per face basis, so the result will in any case (out of a custom shader of course) be smooth and interpolated between vertices, by default.
    But that's not a problem, keep it, as is.

    ... the cherry on cake follows, within the second step.
    // select you mesh
    // grab the Paint vertex colour tool
    // set Paint option to 'face' in place of 'vertex' >> wich would cause further interpolations between vertices too
    // set operation to 'smooth'
    // finally click on 'flood'

    >> Click as much as you want on flood will soften the result on a per basis face colour
    Gives really nice results :)

    You can also use this method with any painted vertex colour too, but in this case, it can also result in some faces being applied some alpha at smoothing-flood >> in this case, just apply a colour in channel mode to the vertex colour set, unchecking rgb chan, and keeping alpha set to 1. Problem is gone ... but it shouldn't happen with full baked vertex colour sets.

    Maybe have a go with Chameleon vertex painter too. I'm sure you can get the same result, maybe even with some additional artistical tweaks in the process.

    EDIT : forgot to mention also you can now export the colourset directly through the vertex colour paint tool > attribute maps > export


    Cheers.
  • Jeff
    Hello Inanimate

    Tried it on my side this morning cause I noticed 'export' was greyed out when setting the vertex paint tool to "face" in place of "vertex".
    Sorry I missed that point.

    So I tried baking a "custom shader" pass through transfer map, by connecting a mentalrayVertexColors-node to a mia_mat#diffuse slot >> cause you need mental ray registered shader in order to use custom baking.
    But the result is really awfull.

    So I'm stil looking for a solution.
    Will drop a line if I got a good result.
    The question remains challenging !

    Hope anyone's got an idea btw :)
  • Jeff
    Ok I finally figured out some easy trick.

    As vertex colour sets stores RGBA value for each vertex, it's in some way understandable it won't ever export the values "flat per face", cause each vertex should then store 4 * RGBA values for this to happen.

    The solution popes out clearly now !
    Select all of your vertices, then simply Edit mesh/detach component.

    >> You're now able to export plain sharp colour per face, as all of the vertices have now been split into 4 duplicates with the original RGBA vertex colour :)
  • inanimate
    Hi Jeff!

    Thank you sooooo much! The smoothing and flooding method worked perfectly. It's great that I can export the colorset as well. This is such an amazing community here.

    I checked out Chameleon too, it looks useful but it seems like development stopped on it recently.

    Thanks again!
  • inanimate
    Ah sorry one more silly question: How can I bake vertex colors based on only the materials, ignoring the lighting? I tried using only the albedo channel, but that didn't work.
  • inanimate
    Just to clarify, here's a screenshot of before and after for a test bake (attached)
  • Jeff
    Hi inanimate,


    You're welcome.
    There's no silly questions, only ones missing datas, sometimes :)

    - You're now able to export your vertex colour set as a "flatColour-per-face" texture map
    - seems like you mapped a grunge texture (Guessing this is not a procedural)
    "baking vertex colours based on only the materials, ignoring the lighting"
    Do you ask about baking an unlit combination of both through Turtle ?
    vColorSet-multiply/add-texture through single pass Turtle baking ?
    Baking through Turtle, without lighting/shading (unlit), should be done with 'diffuse' checked though, not albedo.
    But if you really want to bake a precomp version of both (vcs/tex), I think your best option is through a custom shader with a ilrPolyColorPerVertex used in nodal-shading, to bake combined diffuse. Just guessing, as I don't have turtle installed.


    Please clarify, I'm a bit confused ...
  • inanimate
    I tried using a Mental Ray bake instead and had the same issue. I've attached before and after bake screenshots with the Mental Ray bake settings. I used a checkered pattern here for the material - I was using a grunge texture (not procedural) before, but it seems to happen with any texture when I bake to vertex colors. Odd, since it was working fine before. I tried resetting my Maya prefs but got the same behaviour. I wonder if I had changed some setting to make it work earlier.
  • Jeff
    Ow, okay ! Thank you for the clarification.

    MR bake to vertex settings are correct, but baking your texture into a vertexColorSet, through MR Light&Color mode will work better with a Surface Shader. The surface shader totally ignoring the lighting, you'll get the "color" only, not the "light & color".

    Now, the transfer between texture and vertices will obviously always evaluate the surrounding pixels from your mapped texture in order to set the related vertice's averaged color. So the only way to partially minimize the vertex blending artifacts is to subdivide your mesh (like with smooth mesh / continuity set to 0 et disabling smooth UVs).

    But whatever the subdivision density, and even if you detach vertices components, the baking will still need a smooth-flood in order to have these flat colors per faces we talked about earlier.

    You could, if you don't want to upscale final mesh density, do the operations on a subdivided version (like the one in the bottom of screenshot), then transfer attributes from the high density colorset to the low density one in world space. The low density version will then be crispier BUT the limit-faces for color areas will still blend between vertices in any high res version case.

    Side note for lowcost pixel art, if you intend producing textures for Minecraft for example, and you decided the res to be a 64*64 per face, then work with a 64*64*64 cube ... but of course, it will be impossible to store same info into a 16*16*16 vertexColorSet

    Hope this helps :)
Sign In or Register to comment.