Nasty AO results with Max 2008

Jesse Moody

Ok so I made the switch to Max 2008 at work and trying to render out my AO maps using mental ray. I am pretty sure everything is set up correctly but I keep getting shitty results.

There are no overlapping uvs on this mesh.



Any ideas please let me know.

Mark Dygert

Intersecting low poly geometry?
Update video drivers?
Time to use a sky light, switch the material color to white and render a diffuse/Fake AO map?

Dunno really, haven't run into that before, weird...

Marisa

I don't have 2008, but everytime I get weird rendering problems like that I reset Xform and collapse. Try doing that.

Jesse Moody

Reset xform and the likes... ended up just going with the scanline process...

MoP

i usually just use the scanline renderer with a skylight, but I've rendered a fair few AO maps out of max 2008 using mental ray without any problem.

how are you doing it, do you have a dirtmap/ao map in the material? That's how I did mine. Also might wanna check that your lowpoly mesh isn't casting shadows (assuming you're baking high to low) - if you're just baking AO into a lowpoly mesh then I have no idea what the problem is, I've done it several times on meshes with overlapping UVs without any issue.

An image of your problem might help, I currently have no idea what the problem might be because I can't see any examples, and you just say "shitty results" which is hardly useful, and could be caused by anything.

Jesse Moody

Sorry mop i must have deleted the pic.

MoP

So, it is a high to low bake, then?
Have you tried turning off the shadow casting properties of the lowpoly object?
It almost looks like a ray distance error... are you using a casting cage, or specifying ray distance?

Mark Dygert

If you are baking high to low and errors are being caused by the low casting shadows, or intersecting, you can also select the low poly, go down to element and hide everything, (leaving it selected) go bake it again.

Also for the skylight method, if you find the skylight is slowing you down, try using the E-Light Dome script. Use the light properties options inside the script to control the settings for all the lights before and AFTER the dome is created.

Eric Chadwick

Using mental ray in RTT you have to hide the low-poly faces, like Vig suggests.

Jesse Moody

hide the low poly faces? like how? isn't it a projection onto the low poly?

MoP

Right-click on the lowpoly object and go to Object Properties, look for the checkboxes in there for "cast shadows" (and possibly "receive shadows" as well, not sure), try turning those off on the lowpoly only (yes, the one with Projection modifier on it - Mental Ray is weird).

Mark Dygert

If what MoP said doesn't work, try setting the low poly to be non-renderable, same right click/object options window MoP pointed to.

IF that doesn't work...
- Select your low poly object
- Under Edit Poly go into Element
- Select all
- Under the Edit Poly Geometry rollout, click "Hide selected" (It will keep the low poly from being taken into account when rendering).
Don't deselect the object but if you do, select it by name (h)
- Bring up the RTT window and render your AO.

Jesse Moody

wow weird...ok i'll give that a shot guys. Thanks....

Something new this mental ray is to me

j_bradford

I don't have Max near me, but my workflow for setting up an MR render is as follows;

Set your renderer to Mental ray (duh)

Apply an ambient occlusion pass to your diffuse of the material

Samples anything from 100-300, spread around 2.5

Right click on your lowpoly, properties -> object ID should be a positive number, we''ll say 7

In your ambient occlusion material.. in the incl/excl field, set that number to -7, this excludes the lowpoly from being rendered down/intersected in the scene

Apply your MR material to your highpoly

Select lowpoly, apply a RTT property, give it a complete MAP for the AO.. along with the rest of your settings you apply in RTT (fairly standard stuff).

Hope that helps, I think thats all I do.

Geezus

I've had similar issues in the past. I've found that simply hiding all the low poly faces fixes the problem, like previously mentioned. For some reason, though, hiding all the elements didn't work. I had to hide all faces, not elements, and that solved my issues.