Alright so I've seen people complain about using AO multiplied onto their skin materials for characters, and I've seen some people mention doing something else, like the multiply blend mode is the problem. In reality you simply need to do some quick edits to have usable AO for skin.
This is a really super quick method, using the Gradient Map function in photoshop.
Just add a gradient map, clipped to your AO layer, and whip up a nice little skin shading type ramp. Then mask off the areas of your texture that are not skin, in this case, the eyes. This is a really quick and non-destructive method, so you can easily experiment with different values.
This method could be used to quickly alter the values of AO for other materials too, so play around with it.

I'm not really a character artist or anything, just sick of seeing people talk about AO like its "bad" for characters.

We just need to apply some basic art theory here to the AO, just as you wouldn't shade/paint shadows with black for skin in tradition art, you wouldn't multiply straight black on your texture.
Replies
I find this hue gradient of the AO map (instead of just the single colour multiplied by darkness that you'd usually get) gives almost a fake depth to the skin, the flatter areas are a bit more 'tanned', and the other areas are a bit more fleshy.
http://tutorial.jcwcn.com/3D-Graphics/Maya/Modeling/2008-08-03/9822.html
by pasting the green channel on top , It lightens it up a bit, avoiding that dirty grey look.
also here
http://donaldphan.com/tutorials/xnormal/xnormal_occ.html
works quite well by lightening the dirty darker areas
Pesonally I use the ao a guide to paint 'under' and sometimes do other stuff which is supposedly technically wrong, but looks nice anyway
The problem with colorizing your ao I find and I have tried this approach in the past is that it gives your texture a monochomatic tinge which is hard to get rid of. I still think the best thing to do is to make the skin texture underneath look damn good whether hand painted or photosourced, but using ao as a final touch to give a 'little' depth to accentuate forms gently rather than just blatting it on.
Yeah, this is what is nice about the gradient map method, you can easily assign different color gradients to different brightness values of image, to get rid of the monotone type coloring.
This is a really bad workflow to rely on, because:
1. Its uv dependant, so your uvs need to be orientated in a specific way for it to work, resulting in a restrictive way to uv, or a pain in the ass to "fix" the "ao" content.
2. You're baking smoothing compensation into your ao, which is always a terrible idea, its like adding smoothing errors into your diffuse.
3. Its a poor representation of an effect that is easily created "for real", just add a plane or hemisphere below your mesh.
A gradient map can, or a simple curves tweak can do the same, without having to spend time doing test bakes, and purposefully reduce the range of your image, which is bad for flexibility.
Again, gradient map solves the monochromatic issue.
I don't know if I would argue what is technically correct over whatever gets a person a good end result. Normally its the persons artistic skill and ability to refine just about anything into something amazing, that sees them over the finish line. Obviously Ruz doesn't have a problem refining his techniques into amazing looking faces so I think it underscores the need for a traditional background over technical correctness.
Typically people who end up following a technique without really knowing what end result they are driving toward, end up frustrated and confused when the specific settings in the example don't work for their situation. They get mad blame the technique and don't realize it had more to do with their lack of understanding, which is why I'm glad you said...
I think you're right. When it comes to painting faces, I think a more traditional background will see someone through than a few technical tricks. Not that these tips won't be helpful, but that hopefully the person already understands how to get to the same end result and these are just helps them get there faster.
Its like bragging that you "didn't use reference".
Being technically sound is a skill that will benefit every artist, character, prop, whatever. Technique isn't something to replace artistic skill, rather it compliments it. I've seen very many extremely skilled artists have very messy work because they do not understand the technical side. Artistic ability will only get you so far.
1. Rendered standard ao
2. Rendered ao with a hemisphere below mesh(much bigger than the example, but scaled down for illustrative purposes)
3. 50/50 blend
The two bake method here makes it easy to tweak the % to get the desired amount of the more "directional" ambient bake that the hemisphere method gives us.
If my AO ended up looking like the bottom right image, ao+green channel of NM, I dont think I would be happy to use it either, and instead try to paint "under" it. But the much better solution is to just simply make GOOD ao content.
No matter how artistic you are, you can see the the technically correct method simply looks better.
I tend to use AO or cavity as a guide to paint colour.
TBH though I am going to do some more tests as I am not discounting what you are saying EQ, its probably just been the case that my AO maps are not of a high enough quality to use as I would like them
The usual non-directional AO seems to work best in most situations because the areas you'd expect to be darker always are. If you use directional AO in addition to that, any time a character moves, you're going to notice areas that should be lighter.
It's super handy for static models and weapons, though.
There is a reason this is posted in tech talk, and not *pretentious artistic wankery*, you can paint your normals by hand per channel in photoshop, you can do all sorts of weird shit to your mesh to get "good" bakes, but clearly someone can say this is poor technique. If the person provides a convincing argument to show that it is infact poor technique, then they, sir, are the ones who shall say it is. End result does not = technique.
You could print your uvs, and paint over it with oils. Certainly this would be very artistic, and could even produce some really cool results. Who's to say this would be poor technique? Anyone with a brain.
This idea that no one may question an artists methods if it gets a certain result is self destructive and very limiting.
Unless your characters are constantly standing on their heads, I think some directional AO is nice, certainly my example could be heavy handed, but in the specific workflow outlined, it would be very simple to tone it down. I feel without a hint of directional AO, the end result feels flat.
Still, it seems that standard AO tends to look fairly directional on heads anyway because of how they're constructed. The bottom of the jaw tends to be occluded by the neck and chest, for instance, so it's naturally darker than surrounding areas in standard AO unless the sample distance is small.
Yeah that is all I ask, a feel there is a big misconception with character artists and AO, that stems from poor AO content, or not knowing exactly what to do with the ao.
So I have a request here, since I'm not much of a char artist so I dont have a lot of content sitting here, is someone willing to donate a high/low full character mesh to the cause? Ideal something like a simple monster or human, that doesn't have too much/any intersecting props and fanciness to muck up the process. Then we can all take a crack at specific AO methods, and post our settings in various apps and whatnot.
This is actually very similar to what I do with AO on weapons etc. Parts that animate need to masked off. Same goes for characters I guess, looks like its not so different after all! =P
I also like having a bit of directional AO baked in, but it really depends on the projects.
When working on photo-real stuff, with lots of cut-scenes and fancy lighting, it's usually best to have barely any at all, let the fancy skin shaders make things look pretty (obviously you need a proper shader, and a robust lighting solution).
Also, people tend to die a lot in video games (so much violence!), and end up on the floor as corpses, directional AO can look really wrong in that situation sometimes.
But yeah, in the end if it makes your character look better 90% of the time, it's probably worth doing!
This very common idea that if an artist reaches a certain result his methods are above critique is nothing if not pretentious artistic wankery. There is no "nerd rage" here, just stating simple facts. This is less directed at you than it is the idea of such.
Your technical points here are extremely dependent on what the art director and publisher wants as a final result. Emphasising FINAL RESULT, I find you guilty of Pretentious Technical Wankery. Very common in the industry.
This directional+ao look is an overhang from painted textures, I`m very surprised it appeals to you, it is in opposition to the work a normal map and mesh does! I'd expected you to be more forward looking. I absolutely hate the look, I'm forced to use it on a project at work... by blind fucking idiots!
Aiming for realism? Then DO NOT use AO on flesh at all, SHADING should come from SHADERS.
This thread... Subjective, OVER RULED.
I just found it too time consuming.
I am certainly going to try some more AO stuff though, some good suggestions in this thread, insults not withstanding:)
I think you're missing the main point here, Ruz says he uses the green channel ontop of ao, points to a tutorial that says use the green channel to get "directional ao", and I show a method how to get high quality direction AO instead of using the smoothing-error-resulting green channel method.
I also show a simple way to blend between standard AO and directional AO, that should suit the art style or tech of any project, never stating that "you must use directional ao!".
Of course the complexity of your lighting, and overall art style will have final say, and I never said anything to the contrary.
Oh, this is also entirely bullshit, unless you have realtime bounced radiosity, which admittedly is becoming more common, however unlikely that many people here are using an engine that does. Its important to note that skin shading, and occluded light are two entirely different things, and are not interchangable.
Using heavy AO might not be advisable for hero characters, but it could be very useful if you have a large number of NPCs that are going to be fairly static. A good example of that would be in an RPG with towns full of NPCs or a game like Assassin's Creed with crowds of people all milling around. If you have 50+ characters onscreen at once, it'd just be silly to have high-quality skin shaders for all of them because something like that could bring even a hefty system to its knees. In a case like that, baked textures would be the way to go.
This actually makes much more sense than grabbing any info from a tangent space map, as OS isn't relative to your low's mesh normals. OS map data is consistent, meaning you can use it to pull certain angles regardless of uv direction. We use OS maps for some texturing stuff at 3ps, like direction specific effects.
Is there an advantage to using a gradient map adjustment layer instead?
Yes, the main advantage is that you can use a gradient to define different colors in relation different brightness values, for instance you want your highlights more yellow and your shadows more red. That sort of thing, instead of just a basic color value being applied, it can be varied. I actually used to use the "variations" thing in adjustments, to do this sort of thing, but the gradient map is nice because its non destructive, and you can easily save/load presets.
This is perhaps overkill however, and the important this is that you dont throw unmodified gray ao on top of skin, as it tends to desaturate and look dead.
I did a quick test with Mental ray, using a base mesh that comes with silo (the best I had available)
On the left is AO at 64 Samples and infinite distance. it took 1 min 38 seconds to bake.
On the right Final Gather. it took 44 seconds to bake. (mental ray 'low' preset, and 2 diffuse bounces)
both maps are 1024
A better mesh would be more telling. but it like an interesting idea.
yeah it was ok kind of technique, but if you had multiple assets, it becomes a real bind to combine all these different overlays and even to render them out is an extra step in the pipeline.
Also the shading of the OS map channel we used can be dark in one area on light in another, so its not ideal, but we worked around it.
Most clients I work for don't specify which method I need to use as long as it looks good to them.
my last job I used 3 overlays for each texture which looked relly nice, but when your psd has 50 layers x3 overlays it becomes nightmare + revisions take forever
So ideally yeah I would just use a cavity map or an ao map on top of my hand painted or photosourced layer, but if its looking shit I have to faff aorund in a ponsey artistic way until it looks right:)