I need a faster/better way to bake slection maps

BARDLER

Sorry I have another question XD I am trying to speed up my workflow as much as I can.

I need a faster and better way to bake selection maps. Currently I use Maya, apply different colored material to each part of the highpoly I need, and then bake it using transfer maps. The problem is the projection of the selection map usually is slightly different than what I get from the normal/AO from xnormal so I have to spend a decent amount of time cleaning it up. On top of this Maya does this so god damn slow that it hurts me on the inside.

Is there a faster and better way to do this? I have seen that xnormal can read polypaint data, but can it read something similar from maya? or could I hack a way to quickly get my color information from Maya to xNormal? I was thinking a good alternate would be to export chunks of the highpoly into zbrush, fill each subtool with a color, and export to xnormal. I would like to avoid that if possible though.

Thanks again PC!

passerby

poly paint is just vertex colour which you can export out of Maya to xnormal and bake just like polypaint

BARDLER

Holy shit balls I did not know that. I just did a quick test it worked perfectly! Thank you so much.

If anybody stumbles upon this thread then just a quick tip if you do this.
Follow this video, except do color/lighting instead of AO. https://www.youtube.com/watch?v=RY3uzEQjwDU
Maya has no way to bake color without lighting in the bake vertex color menu. So all you need to do is use surface shaders on your highpoly because they hold no lighting information.

chrisavigni

Hey Bardler,

Maybe you can use this:
http://www.polycount.com/forum/showthread.php?t=92535
You can apply random colors to uv shells and use as selection map.

SlyRipper

You could also use xNormal for this. If you don't have a colormap imported to it, then xNormal will use basic colors you'll define. Mostly I have black and white selected and you get kind of a mask after baking the diffuse/colormap. Those will be the exact same as you already created from normal/AO and what not. If you want some colored stuff you can easily fill those white islands in photoshop or another tool you're using (except the colored parts are melt together, then you probably need to bake them several times and use them as layers)

passerby

@badler you a actually can just bake the vertex colour from Maya with xnormal too, just export fbx or sbm, and uncheck ignore hp vertex colour in xn and select vertex colour for map type.


I do it all the time and even got a little script and ui for applying preset colours to different polygon selections in Maya.

I also use the same method for bakeing curveature information from Maya since I like the result of a Maya script I use for that over xn.

BARDLER

passerby wrote: »

@badler you a actually can just bake the vertex colour from Maya with xnormal too, just export fbx or sbm, and uncheck ignore hp vertex colour in xn and select vertex colour for map type.


I do it all the time and even got a little script and ui for applying preset colours to different polygon selections in Maya.

I also use the same method for bakeing curveature information from Maya since I like the result of a Maya script I use for that over xn.

That is even better! Thank you very much.

passerby

ya im away from home atm, but can post you the script once im back at my main computer again. Just a quick way to assign vertex colour with 1 click, than you can easily export out and bake with XN, also has a tools for making a map for hte lowpoly that has all UV shells as differnt colours.

BARDLER

passerby wrote: »

@badler you a actually can just bake the vertex colour from Maya with xnormal too, just export fbx or sbm, and uncheck ignore hp vertex colour in xn and select vertex colour for map type.


I do it all the time and even got a little script and ui for applying preset colours to different polygon selections in Maya.

I also use the same method for bakeing curveature information from Maya since I like the result of a Maya script I use for that over xn.

So I just tried this and it didn't work. Here are my steps.
Assign the different colors to each material I want on my highpoly. I am using dDo's swatches for this.
Export out FBX of highpoly, lowpoly, and cage.
Load them all into xnormal.
Uncheck box that says ignore per-vertex color.
Check Bake highpoly's vertex colors.
It renders all white, which is the background color I chose.

xNormal wont actually bake the vertex colors into the mesh from the materials I chose I don't think, unless I am missing something. I will still have to do that in Maya, which is annoying because Maya has no option to do that without the lighting.

m4dcow

BARDLER wrote: »

So I just tried this and it didn't work. Here are my steps.
Assign the different colors to each material I want on my highpoly. I am using dDo's swatches for this.
Export out FBX of highpoly, lowpoly, and cage.
Load them all into xnormal.
Uncheck box that says ignore per-vertex color.
Check Bake highpoly's vertex colors.
It renders all white, which is the background color I chose.

xNormal wont actually bake the vertex colors into the mesh from the materials I chose I don't think, unless I am missing something. I will still have to do that in Maya, which is annoying because Maya has no option to do that without the lighting.

The workaround is to crank the ambient on all the ddo swatches up fully so no lighting info is baked. It's a PITA, but you can just do it once in the swatches file and then you have it for use later on.

Then do a vertex bake of lighting and color, and the vertices will have the colors baked.

BARDLER

m4dcow wrote: »

The workaround is to crank the ambient on all the ddo swatches up fully so no lighting info is baked. It's a PITA, but you can just do it once in the swatches file and then you have it for use later on.

Then do a vertex bake of lighting and color, and the vertices will have the colors baked.

That is a smart idea, but yea a pain in the ass. It is so weird that Maya can't bake vertex colors without lighting information. I wonder if it would be faster to convert all the dDo swatches to Surface Shaders in the hypershade instead. I will mess around with it tomorrow, or maybe Passerby as some crazy script for that lol.

Edit. My girlfriend come to the rescue. She wrote this for me for Python. It switches all the ambient colors of any selected materials to white.

import maya.cmds as cmds
sel = cmds.ls(sl=True)
for mat in sel:
    cmds.setAttr("%s.ambientColor" %mat, 1,1,1)

Changing 1,1,1 to 0,0,0 will change it back to black.

pior

If you are still running into the issue of different bakes from different programs not aligning correctly (especially xnormal compared to maya), try to compare the results of such bakes with and without a cage object. (the cage object being a simple inflated copy of the original low, done using "move components" along the normals). This might solve your problem.

BARDLER

pior wrote: »

If you are still running into the issue of different bakes from different programs not aligning correctly (especially xnormal compared to maya), try to compare the results of such bakes with and without a cage object. (the cage object being a simple inflated copy of the original low, done using "move components" along the normals). This might solve your problem.

That is only one issue though. My major problem is how long it takes to actually bake the map

passerby

it wasn't working since you just applied different materials to the faces via the ddo materials, my method is still different. works with vertex colour, than xn bakes. if I have time after wrk today I will dig up the script I made for this.

its something I made that allows me to quickly assign vertex colour to face selections, and to visualize this.

BARDLER

passerby wrote: »

it wasn't working since you just applied different materials to the faces via the ddo materials, my method is still different. works with vertex colour, than xn bakes. if I have time after wrk today I will dig up the script I made for this.

its something I made that allows me to quickly assign vertex colour to face selections, and to visualize this.

Ahh I see. It might take some time setting up, but as long as the colors for the the ddo materials are the same as the vertex color I assign it should work perfectly fine. Thanks again! Can't wait to try that script out.

passerby

ya in the process of tweaking it a little, and making it use ddo colours, will post what i got tomorrow.

it has its pros and cons over the material method

pros:
Simple baking process with XNormal.
doesn't clutter up scene with materials.
easy to apply.

cons:
is per vertex not per face, so there is a little tiny bleed with interpolation between verts.

ghaztehschmexeh

BARDLER wrote: »

=
Check Bake highpoly's vertex colors.
It renders all white, which is the background color I chose.

Did you uncheck this? :P

EarthQuake

BARDLER wrote: »

That is only one issue though. My major problem is how long it takes to actually bake the map

Yeah, Maya bakes are single threaded and *embarrassingly slow*.

If you're using mesh->smooth for a sub-d object, what you need to do is this: save out a "dirty" copy of your scene, and delete all history, this will freeze your mesh in the sub-divided form and will likely significanlty speed up bake times. I've had scenes where it was taking like 7 hours, to bake a NORMAL MAP AND COLOR MAP, and when I did this it was like half an hour, which is still retarded slow, but atleast workable.

BARDLER

EarthQuake wrote: »

Yeah, Maya bakes are single threaded and *embarrassingly slow*.

If you're using mesh->smooth for a sub-d object, what you need to do is this: save out a "dirty" copy of your scene, and delete all history, this will freeze your mesh in the sub-divided form and will likely significanlty speed up bake times. I've had scenes where it was taking like 7 hours, to bake a NORMAL MAP AND COLOR MAP, and when I did this it was like half an hour, which is still retarded slow, but atleast workable.

What really sucks about it is that my face looks like Fry's.

Not sure if frozen, or should wait longer.

The work around I have now is pretty fast. I assign surface shaders with the colors I want and bake the color to the vertices in Maya. I did a test last night and it took about 3-4 min to do that in Maya on a smoothed 2 million poly mesh which isn't bad. Then bake the color map in xNormal which takes about another 3-4 min to do.

Passerby's script sounds nice if I can get it to work with the dDo swatch colors easily. In the mean time though my girlfriend wrote me a script that converted all the dDo phong materials in Maya to surface shaders that have the same RGB value and name. Super helpful.

Edit: Script if anybody wants it.

import maya.cmds as cmds

newMats = []
sel = cmds.ls(sl=True)
for mat in sel:
    # Query name, color, and shading group.
    matName = mat
    matColor = cmds.getAttr(mat + ".color")

    matSG = cmds.listConnections(mat + ".outColor", d=True, s=False)
    cmds.delete(mat)
    # Create Material
    print matSG[0]
    newMat = cmds.shadingNode("surfaceShader", n=mat, asShader=True)
    # Apply color
    cmds.setAttr(newMat + ".outColorR", matColor[0][0])
    cmds.setAttr(newMat + ".outColorG", matColor[0][1])
    cmds.setAttr(newMat + ".outColorB", matColor[0][2])
    # Connect newMaterial to the original material's shading group.
    cmds.connectAttr(mat + ".outColor", matSG[0] + ".surfaceShader")
    newMats.append(newMat)
cmds.select(newMats, r=True)

Credit: http://sundaram3d.weebly.com/