How to use pre-rendered AO Map with 3ds Max

TorQue[MoD]

Hey all!

I've been trying to figure out how to use a custom made AO map that I exported from Mudbox with my scene inside 3ds Max 2010. I'm not sure where I would use the tga inside the material editor. I've got my unwrap on diffuse color and my normal map on bump... where would AO go and what material type would I use? The Ambient/Reflective Occlusion material seems to be for dynamically casting AO onto other meshes in the scene, not to overlay a custom made AO onto your mesh. I tried looking around for how to do this with no luck.

Thanks!

Spicypixel

Forgive me if I misunderstood but isnt the AO meant to be baked down to the diffuse, possibly as a layer in photoshop?

Bubba91873

Yeah the ao is used as a layer in photoshop set to multiply.helps liven up the texture.

r_fletch_r

If your presenting low poly work you'd be better off using a good viewport shader or the marmoset toolbag. Rendering low poly with mental ray totally defeats the point.

In 3point and maromset you use the AO for masking the ambient light, which gives you nice looking fake indirect lighting. Other than that as the others said combine it with your diffuse to boost the details.

You can invert it and use it as a layer mask in photoshop. this makes a great starting point for dirt (all the crevices and corners get dirt)

TorQue[MoD]

Sorry, maybe I didn't explain properly... I've exported an AO map from mudbox for use in game with the UDK and I was simply trying to test how much of a visual difference it makes when used with my low poly mesh. (Trying to get a feel for how much better it looks) I thought you'd use it in the same way as you do a normal map... apply it to the model and get better details without actually having to do the processing of true light occlusion in real time. Sorry if I'm mistaken... I'm really new to AO maps in general.

Ark

If it's going ingame, then you'll have to bake it into your diffuse, normally using multiply works the best.

CheeseOnToast

Using it as a multiplied layer in PS is definitely the easiest, but also not the prettiest. Simply adding black to crevices etc. can look pretty bad, particularly on skin. Much better to use it inverted as a layer mask as r_fletch_r said. At the very least tint it to get a bit of colour in those shadows.

TorQue[MoD]

Oh ok, I thought it was used like a normal map where you blend it together with the diffuse. So this is just to improve the look of the diffuse texture by blending it it for the extra details... Interesting. You think there'd be a way to just assign it to another material slot and have it auto blend like a normal.

Thanks guys!

Ark

There is a technique where you can add the AO to the normal map, but it think it requires support in the shader.

Basically your faking indirect light by baking it in the diffuse, so id keep it subtle to avoid the cell-shaded look.

leechdemon

I've heard of engines that can use AO maps and take them into consideration when being hit with direct light, but I don't know the best way to test that in Max; I imagine a composite using Falloff or something.

I always just use Composite and set the layer mode to Multiply inside of Max. You get double shadows though, so you either need to turn up Self-illumination or put a Skylight in the scene in addition to whatever other lights you want to use.

EarthQuake

CheeseOnToast wrote: »

Using it as a multiplied layer in PS is definitely the easiest, but also not the prettiest. Simply adding black to crevices etc. can look pretty bad, particularly on skin. Much better to use it inverted as a layer mask as r_fletch_r said. At the very least tint it to get a bit of colour in those shadows.

Or simply edit the content of the AO map and use multiply, there is no rule that states AO must be multiplied without editing. A simple gradient map to act as a "skin shading ramp" masked to the materials that have skin works well.

leechdemon

For organics, you might want to try something like color burn instead, or do like Earthquake said and use the AO map to modify another layer, like through a layer mask or something.

TorQue[MoD]

Actually what I ended up doing was Invert the AO layer then use Difference as the blending mode. This made it pop out a lot more than just using multiply as you can see here:


The final result of my model (watch in HD):
[ame]http://www.youtube.com/watch?v=PqZ8PwjHqEU[/ame]

Thanks for all the help guys. Its really appreciated!

EarthQuake

leechdemon wrote: »

For organics, you might want to try something like color burn instead, or do like Earthquake said and use the AO map to modify another layer, like through a layer mask or something.

Not really what I was saying, I made a quick little post about it here;

http://www.polycount.com/forum/showthread.php?t=81562

TorQue[MoD] wrote: »

Actually what I ended up doing was Invert the AO layer then use Difference as the blending mode. This made it pop out
a lot more than just using multiply as you can see here:

You can get the same result by tweaking the curves(contrast) of your ao with multiply. C'mon people, dont be afraid to edit your AO.

Multiply is good because it is a reliable, predictable method, that you can use on all of your assets. It would become a bit of a mess if you did something different for your AO every time you made a new texture.

CheeseOnToast

Or simply edit the content of the AO map and use multiply, there is no rule that states AO must be multiplied without editing. A simple gradient map to act as a "skin shading ramp" masked to the materials that have skin works well.

I use a gradient map on my AO/cavity maps all the time. Great for getting a quick base with some decent colours very quickly. You tend to end up with a handy library of gradient ramps as well when you work like this.

arshlevon

It's just information and can be used in many ways. I have used the gradient map approach, i also like to take a copy of my diffuse and overlay it on top of my AO. This gives you a colored AO map and looks much better when multiplied than straight back and white. I am not a fan of how AO looks just multiplied, kind of makes it look dirty, i try and shy away from pure black in my diffuse. For different materials sometimes i do different things with the AO. For skin i just use a hue sat adjustment to color it pinkish purple, lighten it a bit a multiply that. also inverting it and using it as a mask for adjustment layers is nice, that way you can tweak stuff on the fly and if your base map changes colors it's a bitt easier to go ba and adjust.