3dsmax ambient occlusion bug?

Delaney King

This is a new one that cropped up today and is killing me.

I need to render ambient occlusion using render to texture. No problem so far. Ive done this a lot before so I know the steps.

Weirdly, my results today show black wherever the low res model intersects the high (?????!!!!) This would mean that mental ray is now including the low res mesh in its calculations. AAHHHHHHHGGGGGG!!!!!
Is this just me or is this now a critical bug?

Palleschi

I only know how to cast a GI map.. no idea how to do AO.. post yer process and Ill give it a run, and see if I get the same problem..

JordanW

as it stands you cant use MR to render AO on a projected model, you have to use skylight + advanced lighting and output a lighting pass using max's scanline renderer instead of MR

ScoobyDoofus

Yeah...IM with these guys. I've never been able to do an AO bake in max using a projected model.

EarthQuake

Just use skylight with lightracer

Xenobond

Render to texture works fine in Mental Ray. when you have it set as your default renderer, an ambient occlusion map(MR) is added to the types of maps you can render out as.

I'm pretty sure you must have tried resetting xforms/checking your normals, right?

Silva_Spoon

Dave I got this same error before. Never got passed it. Drove me INSANE!!!

KRakarth

I have to say I generate my AO maps using a skylight and light tracer as well.

I seem to vaguely remember having the same problem with rendering AO maps with mental ray myself as well.

Ruz

hmm I read of a workaround for this. something about changing material id stuff the in the mental ray shader. its on the UDN if you are an unreal engine 3 licencee

FAT_CAP

Yeah theres a workaround someone posted on UDN - baking the Mental Ray material into the diffuse slot or something...can't remember the exact steps but the results were almost identical to adding a skylight and lighttracer to your scene...only more unwieldy to set up!

Noren

You can exclude object-ID's with the AO-Shader, the Render To Texture dialog misses that option unfortunately. By putting an AO in the diffuse-slot of your source geometry and setting SelfIllumination to 100% you can bake an AO-pass and still be able to exclude single objects. That means of course that you can't bake diffuse and AO at the same time. Would be great if there's a better workaround like excluding objects from being visible to AO in general.

zweek

Same bug is in Max9, drove me crazy for hours.

MoP

OLD THREAD ALERT!
Question: Why is everyone using mental ray to bake AO when the skylight + advanced lighting solution works fine? You can set up all sorts of nice bounce lighting and tweak the settings in there for a nice fast and high quality bake, and it doesn't take ages to set up - just add a skylight, set its colour to white, turn on Adv. Lighting, set the rays down from 250 to 50 or something and bake it... I haven't had any problems with that.

Is there something mental ray does noticeably better or faster?

Xenobond

All I have to do is set the renderer to mental ray, and select the AO map to be rendered in the render to texture dialog.

arshlevon

to tell you guys the truth, i actually get much better results using an oldschool light dome setup.. there are several scripts for this.. choose the one you like the best, but for some reason i find the light dome results seem to catch the details better than the skylight.

Ruz

yeah i got this and gave up on it really quickly. TBH amb occ maps suck anyway. trying to overlay one on you diffuse map is akin to painting charcoal edges on your colour work

Xenobond

Yeah, I totally agree. But the minute spent on the ao map helps me out greatly when I'm painting the texture in bodypaint in a fullbright environment.

rooster

I found that putting a negative push on the lo model before baking fixed it so it was beneath the hipoly, since the object id exclusion seems a bit dodgy

Toomas

Or you could perhaps make the lowpoly invisible or non renderable?

Jaco

Yeah, you could right-click on the low-poly, go to properties and turn off the cast shadows option? Works for me.

artrynk

->Toomas-Jaco - you right
1-When bake Ambient Occlusion with mental Ray Occlusion shader for light(NOT Global Illum. like lightracer do, ITS DIFFERENT Things) in objects properties of your low-poly UNCHECK in Render control all radio buttons exclude Renderable(if ucheck- bake to texture will fall everytime)
Your low-poly will renderable but not visible in render frame window(is it bug of MR working with BakeToTexture?)
2- Ambient occlusion is procedurical shadow, its show places where light is attentuate(fall off?? Sorry my english Im from russia))) and its work something like dyrt, its not lighten deep places even light point directly cast light to it, GI instead can lighten deep places of your object.
AO generated by every part of your model? in max you can setup spread angle, the more rays casted from every point of your model hit surface- the more darkly this part of surface. Flat surface with spread <0.9 will bright, more 0.9 it will self shadowing(dont use more than 0.88)
Thats why Ambient Ocllussion more preferable then GI type of light.

Delaney King

I forgot about this thread!

Ok, the way to do this is to use the G-Buffer.

Set your low poly objects G-buffer to, say, 15.
Then in your Ambient occlusion map set the include/excl. Object ID to negative 15 (-15).
This will then exclude the low poly from the shadow solution.

The previous suggestions about turning off object property renderable etc does not work.

Hope that helps anyone out there from pulling out their hair.

KevinCoyle

So do you use an actual mental ray shader with ambient occlusion in the surface slot to render your ambient occlusion map? I've never needed to use a material to render an ambient occlusion map using render to texture. Do you apply the material to the lowpoly or the highpoly? I've tried to render an ambient occlusion map this way but it just comes out blank.

newton

i know this is really old, but i had the same problem and finaly got the answer to the problem...

the key is that the cage can not be reset. it needs a value over 0. OR you could use
an offset in the optons setup of the "render to texture" dialog.

here is a nice tutorial: http://www.ai-dallas.com/student/Haydel/RH/uploads/Ambient%20Occlussion%20Maps.pdf
mabye this still is interesting for some people..
cheers

GIS3000

Hi there Newton,

Could you please elaborate? The link you gave no longer works. How do give it a value above 0? The Push amount always goes back to 0. The offset option also did not seem to work for me.

I've tried so many methods but all I'm still getting is ray misses (red spot) on my baked AO texture. I've been using the skylight scanline method for a while now, but there are situations where MR's AO is better. So that's why I'm re-exploring this option.

I've tried this
(http://forums.cgsociety.org/archive/index.php/t-560767.html) but still had the red errors.

I would also like to know how King of Daveness used object ID to solve it. I can only find the option on the mental ray AO material but not in the render to texture option box.

Any help would be appreciated. The funny thing is that it works with some object but not others.

regards

Noors

yeah the push amount always gets back to 0, but it is applied nonetheless. Just be sure that the cage enveloppes totaly your highpoly. You can activate shaded in the projection modifier to see it more clearly.

The other issue is about MR taking the lowpoly in account while rendering.
The options are in the object properties. I think putting visibility to 0 works, as assign a different gbuffer ID.

GIS3000

Thanks for the reply Noor.

Yep. I have ensured that my projection cage is pushed out far enough. I also tried using the "CompleteMap" baking option and followed King of Daveness's object ID trick. I placed a standard material on the highpoly model and in its diffuse box, gave it the AO material. I gave the low poly model an object ID of 10 and in the AO material's dialogue box, I gave it a -10 for the exclude object ID value.

None of that seemed to work. I'm still getting ray miss red spots. Basically this is the EXACT same problem as I had mentioned here back in 2011 (http://www.polycount.com/forum/showthread.php?t=91576)

It's as if the baking process no longer listens to the projection cage but to the geometry itself! It seems to be a problem with the projector and nothing else. I hope Newton could shed some light.

Using scanline renderer with skylight is almost 100% trouble free. But as stated in Lauren's tutorial, intersecting geometry is not anti-aliased. Mental Ray does a much better job in corners and very confined places.
(http://www.laurenscorijn.com/articles/ambient-occlusion-baking)

The funny thing is that it seems that some people on the net seem to be able to get it done on a reliable basis. I was also wondering if this process is less painful in max 2014?

Is the Ambient occlusion (MR) option in the Render to Texture option really that broken?

Anyway, I've attached some files. I hope someone could take a look at them. Thanks in advance.

Any input would be greatly appreciated.

Regards
GIS3000

Noors

Are your objects resetXformed ? Did you try to put lowpoly visibility to 0 ? Mind to post this object file ?

GIS3000

Thanks for the quick reply!

Yes I have reset my Xform.

But after more experimentation, I've been able to solve my issue, thanks to you.

It's the VISIBILITY! Nothing else mattered but that!

Once again, thanks for the help. Baking AO from a highpoly to a low poly using mental ray is now an option!

Regards
GIS3000