High quality AO maps ?
polygon
I'm getting frustrated trying to render out AO maps from 3ds max 2010 to be used in texturing. The quality of my renders are not what i'd expect.Some areas are too dark, details not clear enough, unwanted shadows...
I do have some over-lapped parts but those I can live with and edit.
I've tried using both light tracer and mental ray methods, making sure the backgrounds and such are white, model asigned white material and setting sampling qualities to 128 etc... Takes me about 30 mins to render them out.
I'm trying to achieve a ao map that looks similar to this:
http://i259.photobucket.com/albums/hh295/Bubba91873/AOExample.jpg
But I end up getting something that looks more like this:
http://i259.photobucket.com/albums/hh295/Bubba91873/AO.jpg
I want a nice clean crisp ao with more white color and less darkened areas with details more defined.
Been using these sites as guides to give you some background info:
http://www.eddybrown.co.uk/blog/baking-ambient-occlusion/
http://synthesisgraphics.com/tutorials/occlusion.htm
I watched the 3d motive tut on texturing military vehicles and the ao texture came from that.The tut is pro and should check it out.
I do have some over-lapped parts but those I can live with and edit.
I've tried using both light tracer and mental ray methods, making sure the backgrounds and such are white, model asigned white material and setting sampling qualities to 128 etc... Takes me about 30 mins to render them out.
I'm trying to achieve a ao map that looks similar to this:
http://i259.photobucket.com/albums/hh295/Bubba91873/AOExample.jpg
But I end up getting something that looks more like this:
http://i259.photobucket.com/albums/hh295/Bubba91873/AO.jpg
I want a nice clean crisp ao with more white color and less darkened areas with details more defined.
Been using these sites as guides to give you some background info:
http://www.eddybrown.co.uk/blog/baking-ambient-occlusion/
http://synthesisgraphics.com/tutorials/occlusion.htm
I watched the 3d motive tut on texturing military vehicles and the ao texture came from that.The tut is pro and should check it out.
Replies
Any help in improving the ao method is appreciated. I dont want to use high and low poly model bake method. Just want a nice ao from a model already made.
There is so much bad info on the internet btw.
If you are getting unwanted shadows you may have to set your trace depth so it doesnt send rays out into infinity. Not sure though.. haven't baked AO with 3ds.
Yes I've seen those methods. I cant post the model as that would be ill advised.Its not mine and this is personal work and learning for me.
I dont want to bake from high poly to low poly models as that would invlove me making a high poly model for each bake and thats just not practical if I want to use the ao's as textures for 100s of models.....
When you guys mentoin exploding the model, are you saying for me to split the model along the elements or objects ? I was under the impression the mesh should be attached into 1 object ? Thats what i've been doing.And rendering all objects attached into 1.
BTW this is suppose to be tank models. Not boxes found in most tutorials...Almost every tutorial sucks on the internet and is designed to give speed results more then doing quality work.
The model itself is pretty good quality and I cant imagine that its cuasing the poor render.I'm taking care to hide/delete parts and move over lapped uv's as needed.The rest i'll edit in photoshop.
Describe precisely what exactly you think is wrong with the render.
Too many dark areas and shadows are being baked by all the methods that I've tried.And theres been alot....
The dark areas are covering up too many details out of my ao renders.Areas that should be lighter are not, and i dont know how to increase that with just the skylight in scene and the mental ray setups im using.
If I were to use these as a layer in photoshop set to multiply, it would seriously mess up the base layer that I would use to start the texture process.Places would be too dark with not enough detail brought forth.
In the dvd that i mentioned from 3dmotive, I suspect the instructor used Mudbox to render out cleaner ao's with perhaps some editing in ps.
your either going to have to do a high poly and a exploded bake or spend time trying to docter it in photoshop.
and i wouldnt say all tuts are meant for speedy results i found some good articles on a ton of stuff including AO and normal bakes in the wiki here.
When we talk "quality" of AO, generally, you're refering to noise or other artifacts that can be reduced by more ray bounces, or more anti-aliasing. Anything else is related to your specific asset, not how good/bad X app renders AO.
The biggest difference between the "good" example AO you posted, and your "bad" ao, is the source content of the model, that model has a accurate highpoly model used for AO, with fine details etc modeled in. Other than that, the broader AO forms(which is all you'll ever get with a lowpoly model alone) look the same.
Well, if he's a using skylight method, other programs would offer an alternative. Softimage's AO baking doesn't rely on lights, and I think xnormal is in the same boat. You'd be able to control the bias/gain/spread distance as well to be able to tweak the results more as well to limit where the occlusion appears, if he thinks it is coming out too dark. Geometry is still heavily important though, as are UVs (no overlaps, etc)
Yes, various apps will offer a little control, and slightly difference results, however his basic issue is just fundamental, expecting his AO to offer him something that isn't actually there, and never will be without an accurate highpoly render to bake from.
What you're suggesting, and what he seems to be looking for, is akin to someone saying they suck at drawing, and wanting to know if switching from a pen to a pencil will make them better at drawing.
OP: Modeling high resolution models for the purpose of baking normals and AO is standard industry practice, its best to just embrace it as soon as you can, instead of trying to hide from it.
Yes I totally agree with what you said there, sorry if that came off as a bit snippy towards you, it was meant for the OP more than anything. =P
Yeah exactly.
Like I said the mesh itself is quality and I can move the uv's off to the side in the uvw window.But results are same.
I thought since there was so many talented people here that there could be alternatives that most people dont know about, that would ease my workflow and produce results.
Plus, it seems that I can also edit the ao maps in Mudbox as needed with just paint brush along with the bake settings.Seems to be a much easier process then what 3ds max offers.
It you want to reduce the broad dark areas then use MR AO with a low distance setting.
Yes I know about this setting. I've changed the infinate number 0 to 7.0m and tried at 100m.In my orinigal example pic I had it at infinate 0.
If your tank is modelled to scale then the distance should be about about 10cm
AO is shadow based on proximity. If you dont want the ground to shadow the bottom of the tank then you reduce the distance so that its shorter than the distance between the bottom and the ground.
dude!
Way to take a crap on the rim and put the seat back down on top of it.
There ARE helpful people offering solutions that maybe you should think about.
But you seem confident enough to know what your talking about so, have at it and show us your new and improoved results with mudbox, maybe you'll teach us how to improove our workflows......
I was just fooling around with the distance settings to see if there was a difference, 7m and 100m are just expeirments in the distance.
The model is too scale, so if i'm understanding you correctly, the distance I should set that value to is the meaurement of the tank itself or the measurment from the ground up to the bottom of the vehicle ?
From my understanding all objects in the scene affect the ao render and "could" ? cast the shadows or dark areas I'm seeing.
I've also tried baking them seperately and as 1 object attached.
Beleive me, I am listening to everyone and trying new things. But I will not produce high poly models just to bake ao's. I dont want to waste the next 100 years doing that.
I'm listening and being polite.If I knew the answers I wouldnt be here.
So a very low distance setting of 1m or 0.01m f.e. as r_fletch_r was suggesting in his picture be the way to go ?
Also, there are some pretty talented blokes on this board, but very few mind readers. Until you post pictures of the AO on your mesh along with the settings and results you've been getting, then its a lot like trying to give over-the-phone tech support to a grandparent. I don't think anyone will mind that it isn't your model and you're only using it for personal use, just give credit.
http://i259.photobucket.com/albums/hh295/Bubba91873/Untitled-7.jpg
http://i259.photobucket.com/albums/hh295/Bubba91873/Untitled-5.jpg
http://i259.photobucket.com/albums/hh295/Bubba91873/Untitled-4.jpg
http://i259.photobucket.com/albums/hh295/Bubba91873/Untitled-4.jpg
http://i259.photobucket.com/albums/hh295/Bubba91873/Untitled-3.jpg
http://i259.photobucket.com/albums/hh295/Bubba91873/Untitled-3.jpg
http://i259.photobucket.com/albums/hh295/Bubba91873/Untitled-2.jpg
http://i259.photobucket.com/albums/hh295/Bubba91873/Untitled-1-2.jpg
And no its not my model.Im just using this as an example of what Im working on currently.
I could do a ao render with the map applied to the model if that would also help.
I set the max distance in the rtt to 1.0 hopefully thats what someone else meant by the distance value.
And thank you for all the advice.
This may well be your intention but your coming across like you think you know it all.
What people want you to understand is you wont get a map anywhere near the quality of Alec Moodys if you dont have a high poly. All you will get is what you have shown, but with different tonal values.
So far as your render setting go your going way overboard. If your just baking an AO map then Final Gather should not be turned on at all. Your not baking a lighting map so the FG data is not being recorded. You basically making the renderer calculate a load of expensive data and throwing it away.
if your original mesh is well made like you said it wouldn't take much to make your support edges and slap on some subdivsion surface.
and you would also be able to bake a normal map for nice looking edges and any details that are too small to make geo for in the LP
You have three options:
1) Bake from a high quality, high poly model. This will give you the absolute best quality. If at all possible you should always do this.
2) Bake low poly with a normal map applied. IF your model came with a normal map, you can apply the normal map, copy your mesh and pretend that the copy is your high poly then bake using projection.
NOTE: It will factor in the normal map when creating AO but ONLY if you use projection and 2 meshes. If you only bake using the 1 model itself and no projection you're only going to get the AO from the low poly mesh.
This is going to look worse but in some cases what can you do... /shrug
Also note that if you have a normal map you can use the xNormal photoshop plug-in "AO from Normal map" (installs when you install xNormal) to create AO based on your normal map. This isn't a good replacement for baking from a high poly but it might be a better option than #3 and you might be able to combine it with #2... But its still going to look like ass compared to #1.
3) Use only the low poly. This is going to look like ass, as you've found out really nothing you can do about it.
Because mental ray is so limited in the settings department for AO (samples, spread and distance) it can be beneficial to use a custom light setup and bake a diffuse map with lighting and shadows turned on, I would switch back to scanline rendering for such an operation because mental ray takes too long and gives crappy results. BUT EVEN THEN you are still subject to the 3 options above and the quality they offer. By not having a high poly model to bake your AO from... so you still won't get good results.
Your pipeline is massively handicapped because you don't have a high poly model, there are no magic bullets or mystical settings that would compensate for this just a few things you can try to get slightly better results but they'll still be crap compared to the correct way of doing things. If there was a shortcut, people would take it...
I saw the fg in one of the ao tutorials on the net. Lots of bad info out there.
After a long time on my own trying to figure these problems out is when i came here for advice.
I've tried every method you can think of and it looks like crap.
You are absolutely right.
I already have hq normal maps to work with as these are retextures for the most part.I just wanted to update these models with better looking maps and I wanted a firm ao map to start out with.So much for my hopes in a easier quicker way...
After going back and relistening to the beggining portion of the dvd I refferred too, the instructor did use a high poly to bake his ao from with cage.He just didnt show it, only briefly mentioned it.Why I missed it.
#2 option is a possiblbity as I do have the normals.I did try the add normal map to low poly mesh and try to bake from that but that looked like crap.
I didnt try the copy lp and add normal to it and bake from 2 mesh method.
I may have to break down and do some hp mesh for bakes and or use the 2 model with normals method.But Id have to be selective in which models I do this with.I do have xnormals plugins.
What does it mean to "explode" the models ? How should I split them the low and high poly ? By element, smoothing groups or some other method ?
Would speed things up for me and make it more attractive to try hp bakes.Laying out uv's for a 2nd model would suck but doable.
I could create additonal geometry for the 2nd model and make them more detailed or borrow parts from other models that are more detailed and go that route as well.
This is tempting to me...
Examples of this for me would be:
1. Adding in additonal rivet lines along panels.
2. Recreating hp wheels sprockets idlers.
3. Making the gunbarrrel much smoother by using better cylinder etc.
4. adding in tow cables and hand tools
The problem areas on the models duiring bakes are the dark areas under the vehicles such as the sides and bottom, which really looks like crap. I've been really rigid in only using the skylight and not adding in addtional lights to brighten those parts up perhaps and bring forth some details.
I do need to know if the 2nd model needs uv's or not for added geometry.
I do thank you all for the info you guys provided.It is appreciated even if i'm hard-headed.
If you'll only use the high poly for surface/mesh detail Normal and AO then you don't need to worry about UV unwrapping the high.
Only if you plan to bake the diffuse from the high, will you need UV's and even then they can be anything. It doesn't need to be optimal at all, the UV's can be a tangled rats nest and it won't matter as long as it looks fine on the high poly model.
As for the dark areas, Mental Ray doesn't factor in any lighting when it calculates AO, it only factors in how close geometry is and the surrounding angles. You could stick a hundred point lights under the thing and its not going to make any difference. The AO you bake without any lights is the same as the AO with a sky light, it has nothing to do with the lighting.
You could try moving the whole thing up higher off the ground plane and see if it allows light to scatter under it a little more.
You could also scrap using Mental Ray AO and start using a custom light set up and rendering out a diffuse map with lights/shadows turned on, then adding lights under it would help.
Thank you very much, sounds good.
What does it mean to explode the mesh for bakes ? How to break it up.
Just break out the elements and set a key frame for the starting position. Move the pieces out away from each other and set another keyframe.
http://www.polycount.com/forum/showthread.php?t=53731
What you will need to consider though is how the model is capable of moving. If you bake the AO from a tank turret onto the main body of the tank for example, it could look odd if the turret rotates, thus leaving behind its shadow.
learn to paint values manually and shade it yourself
Baking AO is a way of saying, "I like the way the occlusion looks on this model given this renderer, so i want it portrayed that way in a given engine that will not add occlusion to such detail." If you come to that conclusion, it can be assumed that you are achieving that level of detail in occlusion in some renderer or another, and therefore want it faked in another. If that level of detail/accuracy has yet to be achieved through any render at all, then you can't expect to "fake" it by baking.
In that weapon's case.. 2 bakes were done. One from the exploded hipoly. And one from the unexploded lowpoly.
Let's say you have a trace depth of say 10cm. That means that anything within 10cm will be casting a shadow onto your AO map. This may give you unwanted shadows in areas you don't want them in. Obviously you can "fix" this by lowering the trace depth even more but then you'll get less shadows which may or may not be what you want.
The solution to this is to explode your mesh. That means that you separate parts t hat shouldn't affect each others. For example on the tank you may want to split off the barrel so it doesn't get a shadow along the bottom closest to the base of the turret.
It obviously depends on what you want to do with your model. For something static that won't be animated it may be perfectly fine to leave your barrel right where it is. Your AO map will look less clean but hey, who cares? Final results is what matters and if you can shave off some time to achieve it then why not.
EDIT: The same holds true for any map you want to bake from geometry actually... such as normal maps that someone mentioned. Normal maps can be a lot pickier with exploedd meshes than AO maps.
http://vertexbee.com/rttassist.htm
[ame]
And things like the wheels and tank treads too, even the barrel as it rotates up and down will have AO problems where it connects to the main part of the turret.
But you might want to paint a fuzzy shadow so you have SOME AO there just not a harsh dark line. Instead of drawing the shadow you can bake the barrel with different settings to get something softer.
Or use another blocking mesh for the main body of the turret, one specifically designed to cast the type of shadow you want.
Like cryrid said you need to consider how its going to move and tanks have a lot of moving pieces and you want to make sure your exploding the pieces in groups or individually based on what makes sense not just explode everything just because.
Thers several options that are doable for me but will take some trial and error for me.And more reading...
This should vastly increase the quality of ao maps and would be beneficial to normal map making.
Thanks for the ideas that lead me here.I hadnt thought of it before.
Heres a nice link for what i'm talking about, got this from the stickied thread on modeling shapes in this forum:
http://robbylamb.com/Normal%20Map.htm