Baking Vertex Illumination without using radiosity

frubes

I have a selection of buildings which i wish to bake some AO into using the vertex information rather than baking the occlusion into the textures as they do not have their own unique UV coordinates.

It appears that the only way to do this in max 8 is to bake your vertex illumination using a radiosity solution in the scanline renderer (using the vertex paint modifier).

I was wondering if anyone knew a way to bake the occlusion using mental ray into the vertices. I know how to bake it into the texture but cant find an option for vertex baking. I know this is possible from maya but i cant find a way of doing it in max.

Any help is appreciated

pior

This maybe?
http://www.scriptspot.com/3ds-max/pits-peaks
http://www.scriptspot.com/3ds-max/vertex-color-tools-0

frubes

Thanks for the reply, unfortunately these both still need you to subdivide the mesh in order to calculate the radiosity solution which is really time consuming in comparison to just overiding all of your materials with a AO shader, not to mention it takes a while to get the perfect solution as every model is different. They appear to be glorified versions of the standard Vertex paint modifier. Thanks for looking anyway

Mark Dygert

Is there a specific reason it needs to be baked into the vertex color channel? Maybe I don't understand what you're trying to do but it sounds like you want to treat the vertex color channel like a separate UV channel so it stores shadow info? And you want to keep your mesh low poly without sub dividing it?

How do you plan to use the vertex color channel without a lot of verts for definition?

Color me confused but can you point to something that explains how you do it in Maya?

Eric Chadwick

If you bake from mr into a bitmap, you can then use Vertex Paint to convert the map into vertex colors. You still need unique UVs, but you can just do an auto-unwrap and toss it when you're done.

Anyhow, Vig's right, you still need well-placed verts to pull off vert lighting.

MoP

Vig: Maya has a "batch bake (mental ray)" option, and you can choose to bake occlusion to a map, or directly to the vertex colors of the selected mesh, which is quite nice.
You get the same result using Eric's method in Max, but it is a few extra steps.

pior

I'm not sure I understand what you mean with the need for subdivision. Isn't the following example what you are looking for? That's just one click worth of effort using PitsandPeaks!



Only downside is that it only works with crevices, not for a part of the object hovering on top of another. Still handy...

Mark Dygert

The need for sub divide comes into play with large poly areas, like the vents. Right now its shading the deepest part pretty heavily if he wanted something more like AO where the shadows are more in the crevices then you would need to sub-d so you have more verts to blend with. In the example he gave, buildings, more then likely he would have large polys and verts would be few and far between causing the per vertex coloring to blend over large areas.

MoP, ahh makes sense, I wonder why that isn't a MR feature in Max I guess because you can do the steps Eric outlined, seems like it would be better off to just have it set up like Maya does.

pior

Well yeah but if he wants to avoid subdivision on the buildings, the Mentalray bake to vertexcolor won't make it any tighter either.

frubes

http://www.digitaltutors.com/digital_tutors/video.php?v=815

Thats an example of it working in mental ray/Maya. Its pretty much what im looking for but a way of doing it in max without having to go through the exstensive process of calculating a radiosity solution for each building. (Obviously hes doing it on a high res mesh).

Seemingly when you assign mental ray as your renderer in max there is no way of baking the information out to your vertices.

Vig is on the right lines and the buildings have extra sub'd around the crevices to allow for the vertex blending but getting good results with the radiosity is time consuming.

Thanks for the replies

StefanH

did you have a look at the external programs faogen or xnormal? they can bake AO into vertex data.

I've only tried faogen but it works like a charm.

http://www.rusteddreams.net/faogen.html

http://www.xnormal.net/

frubes

yeah i have seen both of those i have only tried xnormal but you dont have the option to bake to verts in that it only appears map based. Granted it was a quick look though. I didnt realise faogen has a trial version so i may give that a go but i was hoping for a slightly longer term solution

pior

Haha Frubes now I am confused, I have no idea what you want or not want (been using the MR AO bake to vertex so I am familiar with the results you seem to expect). However yuo might want to give Xnormal a closer look as it *does* have the ability to bake to verts (used it last monday) Search for the GPU AO tool in the help docs.

frubes

cool cheers pior, ill take a closer look thanks

jogshy

The latests versions of xNormal(3.14 or above) can bake the AO to a map or also to vertex colors. The xNormal default software renderer only renders maps. The GPU AO tool can render ao maps or per-vertex ao.

If you want to bake it to vertex colors you need to look for the "Simple GPU AO Tool section".



If you press the "generate" button with the "image" tab button selected will render the AO to the output image you specify. On the other hand, if you press the "generate" button with the per-vertex tab selected it will render the AO per vertex into a .SBM or .OVB file (which can be seen in the xNormal 3D interactive viewer or imported into a 3d engine very easy ).

Hope it helps.