Baking AO maps, when is it worth it?

Fuiosg

This post will seem a little scattered brained because it is, but I'm trying to weigh the cost/benefit of baking AO maps. The default post-process AO is pretty good, but it seems like you may want an additional layer of it to help define your forms better.

I suppose my problem is twofold: one is that it seems to take a bit of effort to get a good bake-- being as how you pretty much need to hide certain parts of the model, bake separate images, then composite it all together (I use blender). Not to mention forgo some overlapping UV space. And second, that it requires an additional layer of artistic decision making, knowing how much of it to use, how to bake the lighting etc. Which adds more time to your workflow.

Anyway I'm not sure if i'm overthinking this, or if maybe my whole approach is outddated.  If any veterans could add their 2 cents.

slosh

With PBR methods, baking AO is essential to texturing if you are utilizing programs such as substance, quixel, Mari, etc.  And let's be honest, pretty much everyone is at this point.  Before these programs came along, AO was great for your texture to give it more depth in lesser lighting situations in older engines.  Now, the AO is essential as a masking component in current gen texturing programs.  

Elod.H

My initial thought was that you might be overthinking the process.  I did that in the past and baked out exploded models and what not, but in my production environment (and I will be doing it for personal work as well) the best solution in terms of artistic appeal is to keep the models together (unexploded, high and low poly as well) and bake AO like that. You get nice AO around where components meet etc. Why do you need to pay so much attention to how your AO looks? Generally a "default" Xnormal bake or Maya bake should be fine.

Fuiosg

Thanks for the responses. I've tried default AO bakes from xnormal in the past, unexploded, and didn't like the result. Thought it looked sketchy and certain parts of the model seemed to be projected on top of other parts. But it was probably user error, I'll have to look at some tutorials and give it another shot.

Elod.H

When you have issues like that it's usually cage problems. It should be as tight as possible. I've found that editing cages in XN can be quite tedious sometimes, especially with more complex models. That's why I create my cage in Maya (much easier to edit) and then import that into XN. Thing is that if you bake an exploded mesh AO then you will miss the shadows where components meet and that's a pretty big loss for me.

Fuiosg

I changed my method and had some better results, used an actual hi-poly to bake from, increased the rays, used a better cage.

But I want to get back to this question of when you should bake them-- As Slosh implied, is it just if you're using substance painter etc? I admit to not completely understanding how AO works in terms of masking, though it sort of makes sense. But if you're not using these programs, should you still bother with AO? It seems like you would save quite a bit of UV space by allowing islands to be overlapped; and that for certain art styles, like more colorful and stylized, AO actually makes the lighting look a bit muddy.

slosh

Even when I was using AO for texturing purposes(not for masking), I stopped using multiply as that will definitely make your textures muddy.  You can use gradient maps to interpret the AO as colors.  For handpainted textures, I would paint most of my AO out and basically just use it as a guide to paint over.

musashidan

As Slosh mentioned, the AO map is an integral factor in the PBR workflow. It occludes reflectivity and ambient lighting, and automatically blends out when hit by a direct light source, which makes sense. But it is better to have the map tightened up on the texture as opposed to a larger fallof.

It also pops  the detail on the NM. Grounds things. Adds depth and realism.

Fuiosg

So basically just do it, for all assets. I think I'm getting a better idea now-- Unwrap the model so that only the parts that receive the same shadows are mirrored... bake the whole mesh unexploded when at all possible, unless it has moving/hidden parts, etc. And keep the occlusion close to the edges and intersecting areas instead of spread out.

Thanks for all the replies, just needed that bit of clarification.