Your tool of choice for AO generation?

Mr_Paris

EDIT: Thanks for the many, many suggestions everyone. I can't say thanks enough, you've given me a ton of options to work with! I'll make sure to post up my results soon

Hi, I'm trying to figure out the best way to do an AO pass on a low poly (7000 tri) star wars ship model. A Z-95 head hunter to be specific.

I've used Xnormal's simple AO in the past, it's ok but I feel like I could be getting better bakes as it tends to wash things out. Upping the contrast helps but it starts looking a little funny if you go to far with it.

I tend to avoid Maya's baking system due to it's sheer ungainliness.

So I was curious what you, the polycount community typically use for AO bakes, especially pertaining to low poly models that lack a highpoly to bake down from?

If you have any tips for improving the quality of bakes from Xnormal's simple AO generator I would appreciate it as well!

Thanks in advance.

EarthQuake

Honestly I haven't found a great way to do this in any app. It seems virtually all programs bake some of the vertex lighting in with the AO when using only a lowpoly mesh, or some other weird artifacts.

So one thing I do when using the low AO tool in xnormal is, I sub-divide my mesh a couple times(without smoothing, tessellate its called in most apps?). This will result in less mesh-normal type artifacts.

oglu

mudbox or turtle...

Mr_Paris

EarthQuake wrote: »

Honestly I haven't found a great way to do this in any app. It seems virtually all programs bake some of the vertex lighting in with the AO when using only a lowpoly mesh, or some other weird artifacts.

So one thing I do when using the low AO tool in xnormal is, I sub-divide my mesh a couple times(without smoothing, tessellate its called in most apps?). This will result in less mesh-normal type artifacts.

Ah I see, that's an interesting way to deal with it. I'll try that out. Thank you for the suggestion.

Mr_Paris

oglu wrote: »

mudbox or turtle...

I'll have to test out using mudbox for the process. Thanks.

cryrid

If I only have a low poly, I usually just use XSI's Rendermap as it never requires a second target mesh.

Mr_Paris

cryrid wrote: »

If I only have a low poly, I usually just use XSI's Rendermap as it never requires a second target mesh.

What's your process for setting up a bake in a case like that?

Do you usually setup a plane under your mesh, or do you use bowl shape for light bouncing?

chronic

Topogun makes some beautiful AO bakes - and it uses the graphics card to accelerate the calculations so its much faster than XNormal for example.

EarthQuake

The Simple AO tool in XN uses the GPU as well.

cryrid

Do you usually setup a plane under your mesh, or do you use bowl shape for light bouncing?

Depends on what I'm making. If it's a static object that's just going to be sitting on a flat floor or propped in some corner, then I'll add some similar geometry like that for the AO to take into account in order to get a more grounded effect. If its more of a dynamic object that could move or see a number of different placements/orientations in the scene and the AO is going to be baked into the diffuse, then I wont use any additional geo to help keep it weaker.

Mr_Paris

chronic wrote: »

Topogun makes some beautiful AO bakes - and it uses the graphics card to accelerate the calculations so its much faster than XNormal for example.

Alright, adding it to my list of tryouts.

Thanks!

And yeah as earthquake noted Xnormal does support gpu acceleration which is great

CUDA power!

Mr_Paris

cryrid wrote: »

Depends on what I'm making. If it's a static object that's just going to be sitting on a flat floor or propped in some corner, then I'll add some similar geometry like that for the AO to take into account in order to get a more grounded effect. If its more of a dynamic object that could move or see a number of different placements/orientations in the scene and the AO is going to be baked into the diffuse, then I wont use any additional geo to help keep it weaker.

Ah ok, makes sense, thanks for the tip. Since this ship will be flying around I guess I'll be using method 2 more.

Shogun3d

xNormals

Mr_Paris

kaburan wrote: »

xNormals

Alright, thanks for the input.

arshlevon

i have used all above software, for what you are wanting to do i would say faogen would be the best, there is a free trial, and its probably the easiest one to set up. results are pretty awesome too.

http://www.rusteddreams.net/faogen.html

Mr_Paris

arshlevon wrote: »

i have used all above software, for what you are wanting to do i would say faogen would be the best, there is a free trial, and its probably the easiest one to set up. results are pretty awesome too.

http://www.rusteddreams.net/faogen.html

Oh, nice Never heard of that one before.

Thanks a lot for the link. I'll definitely check it out.

r_fletch_r

You might want to check this out. its like what was posted above but is free
http://www.zhangy.com/main/blog/id/22

in max you can smooth a mesh using its smoothing groups to created hard creases. perhaps maya has something similar.

Shogun3d

Mr_Paris wrote: »

Alright, thanks for the input.

No problem! I know it's been mentioned and you have been using it, i'm stating what I normally use =\

Mr_Paris

kaburan wrote: »

No problem! I know it's been mentioned and you have been using it, i'm stating what I normally use =\

Yeah I guess I've just been worried that maybe I'm setting up my scenes wrong before putting them into Xnormal. It definitely doesn't like low poly stuff much since it'll create a lot of faceting, but I liked earth quakes solution of tessellating the mesh till those are less of an issue.

Do you have any tips as far as setting you use to get the best bake results?

I've been under the impression you have to have a plane floating somewhere under your mesh to get proper light bounce. Otherwise I just crank up the contrast on the AO bake till I get a full spectrum from near black to white. Not the best solution in my opinion. It seems like on default settings everything gets really washed out and you don't really see much AO happening.

Mr_Paris

r_fletch_r wrote: »

You might want to check this out. its like what was posted above but is free
http://www.zhangy.com/main/blog/id/22

in max you can smooth a mesh using its smoothing groups to created hard creases. perhaps maya has something similar.

Ah I didn't see that you had re-edited this.

Maya doesn't have smoothing groups per-se, at least not that I'm aware of. You can harden edge normals and faces individually though, which I would tend to think yields similar results...maybe

I can certainly try it out anyway, looks like a fun tool, thanks.

r_fletch_r

Mr_Paris wrote: »

Ah I didn't see that you had re-edited this.

Maya doesn't have smoothing groups per-se, at least not that I'm aware of. You can harden edge normals and faces individually though, which I would tend to think yields similar results...maybe

I can certainly try it out anyway, looks like a fun tool, thanks.

in my retardedness i posted a max plugin sorry about that.

aajohnny

I use Xnormal all the the time

Mr_Paris

r_fletch_r wrote: »

in my retardedness i posted a max plugin sorry about that.

Ha, well I figured I'd find out once I downloaded it this evening. I had a suspicion it was a Max plugin from looking at the video but I was hopeful

Thanks for trying anyway.

StefanH

i used FAOGEN for a long while.. its really fast.

But since 1.5 years or so i have switched back to mental ray. I find the result superior. I use a high spread value most of the time to give a really nice deep AO effect. I found that faogen downside is the lack of options for the AO.

Also a skylight bake with groundplane can be really nice and gives superior results sometimes.

Mr_Paris

StefanH wrote: »

i used FAOGEN for a long while.. its really fast.

But since 1.5 years or so i have switched back to mental ray. I find the result superior. I use a high spread value most of the time to give a really nice deep AO effect. I found that faogen downside is the lack of options for the AO.

Also a skylight bake with groundplane can be really nice and gives superior results sometimes.

See that groundplane method was one of the few alternatives to Xnormal I was aware of. Since you mentioned Faogen too, I'll put that towards the top of my list for now to experiment on tonight. Thanks.

malcolm

Xnormal for high to low transfer, and mental ray for low to self transfer, but don't use the mental drop down occlusion that thing is broken and hasn't worked in a long time, bake light and colour from an ao shader.

AlecMoody

I don't bake ao from low often anymore but I used to use mental ray to do it. Obviously AO bakes look different than skylight bakes and I liked that I could set a medium size radius and then use a ground plane slightly further away so that I got some light underneath and a little ground fade, but not completely black like a skylight. Some times I have done skylight + radiosity if I have something like a vehicle interior and I think AO will look weird.

Mr_Paris

malcolm wrote: »

Xnormal for high to low transfer, and mental ray for low to self transfer, but don't use the mental drop down occlusion that thing is broken and hasn't worked in a long time, bake light and colour from an ao shader.

Alright thanks for the tips.

Mr_Paris

AlecMoody wrote: »

I don't bake ao from low often anymore but I used to use mental ray to do it. Obviously AO bakes look different than skylight bakes and I liked that I could set a medium size radius and then use a ground plane slightly further away so that I got some light underneath and a little ground fade, but not completely black like a skylight. Some times I have done skylight + radiosity if I have something like a vehicle interior and I think AO will look weird.

Good to know thanks for letting me know your process.

CheeseOnToast

Xnormal with the method switched to "cosine" often minimises or eliminates low poly edges showing up in areas you wouldn't want them to, like the sides of a cylinder for example.

metalliandy

I usually just add the LP model to both the LP and HP slots in xNormal and bake that

shogunato

Faogen is a pretty good app

Ruz

Also a skylight bake with groundplane can be really nice and gives superior results sometimes

works great for me this method, fairly fast too.

r_fletch_r

AlecMoody wrote: »

Some times I have done skylight + radiosity if I have something like a vehicle interior and I think AO will look weird.

Have you tried baking Final Gather for that? even with 5-7 bounces its still faster than light tracer. It lacks the fine detail, but you can get that back with a cavity map.
I did some tests on my current model, a 1k Light tracer/skylight bake with hammersley took me 5 minutes.

1k Mental Ray FG with 5 bounces, and a MR ambient Occlusion map (16 samples distance:1 spread 1) took 3 minutes.


When i bake AO in Xnormal i find i get large splotches in the map, they look similar to photo mapping. anyone experience this/have a fix?

fattkid

Another way that I started baking "AO" maps is to turn on global illumination, with a solid white environment, and bake indirect lighting to a texture. Technically, it's not an actual "AO" map I suppose, but it looks just like one, except I have more control over the bake parameters and quality.

I do this in modo, but it should be a universal technique. What's great about this as well, at least in modo, is I get a great AO map in about 2 minutes.

Hey Chronic - If your reading this - I bought Topogun some time ago, primarily because of the AO bake capabilities - and I was never able to get a decent AO bake out of it. I've seen a few images along the way of Topogun AO bakes, and they looked great, but I could never make it happen. If you could offer up a screenshot or explanation of how you do it, that would be awesome.

Shogun3d

Mr_Paris wrote: »

Yeah I guess I've just been worried that maybe I'm setting up my scenes wrong before putting them into Xnormal. It definitely doesn't like low poly stuff much since it'll create a lot of faceting, but I liked earth quakes solution of tessellating the mesh till those are less of an issue.

Do you have any tips as far as setting you use to get the best bake results?

I've been under the impression you have to have a plane floating somewhere under your mesh to get proper light bounce. Otherwise I just crank up the contrast on the AO bake till I get a full spectrum from near black to white. Not the best solution in my opinion. It seems like on default settings everything gets really washed out and you don't really see much AO happening.

Well it depends on the object. Is it static? If so you can probably be safe to do a Low Poly AO bake setting a plane underneath your object and using Global Illumination.

Sometimes I use both, but for my high poly AO generation I typically use xNormals, up the rays and play with the bias, render real fast on a low resolution (ie 256). Then i'll multiply a Low Poly AO (by this I mean I use the low poly asset in Max, throw a white material with No Spec/Gloss, add Skylight and increase rays per sample, throw a plane underneath about 5m under but not touching, and then Render to Texture).

I know the AO faceting can be nasty in xNormals, some solutions are the way you export your objects. Typically I try to avoid exporting an OBJ while the object has a turbosmooth stacked on it.

When i'm ready to export, I kill turbosmooth and convert to mesh smooth and adjust smoothing, then export so it preserves the full topology (turbosmooth can export out in obj with some wierd topology sometimes)

r_fletch_r

kaburan wrote: »

(turbosmooth can export out in obj with some wierd topology sometimes)

Turn off isoline display. I wrote a little script that replaces export/export selected. It just turns off isoline and then exports. simple but a life saver.

Shogun3d

r_fletch_r wrote: »

Turn off isoline display. I wrote a little script that replaces export/export selected. It just turns off isoline and then exports. simple but a life saver.

Good call, that explains why models are jacked up with Isoline on I only use TS during modeling (performance). I use Meshsmooth before exporting (classic output) and adjust relax function to tighten the model to high rez.

You got a link to your script?

Serp

The best for me is Topogun. Really high quality. I prefer it to max, maya and xnormals. Although I haven't done an indepth analysis or anything.

bugo

turtle, tested almost all, none that I tested is better. Fast, layer baking, merge layers to one map, simply great, self occlusion off, etc. And good quality. Don't use turtle 4, use 5 or up.

Bubba91873

Not wanting to hijack thread but had a smiliar question.

Without using a high poly model, what would be suggested methods for doing AO map bakes something like tank models for pc rts games ?
then bringing it into photoshop set to multiply.

I generally just use mental ray and a flat plane beneath but wondered if there was a better method without having to create the high poly version too.

malcolm

bugo, turtle is really hard to get now as I've recently found out because autodesk bought them and shut the website down. Does turtle do high to low or are you referring to just baking a lightmap using the low?

Mr_Paris

kaburan wrote: »

Well it depends on the object. Is it static? If so you can probably be safe to do a Low Poly AO bake setting a plane underneath your object and using Global Illumination.

Sometimes I use both, but for my high poly AO generation I typically use xNormals, up the rays and play with the bias, render real fast on a low resolution (ie 256). Then i'll multiply a Low Poly AO (by this I mean I use the low poly asset in Max, throw a white material with No Spec/Gloss, add Skylight and increase rays per sample, throw a plane underneath about 5m under but not touching, and then Render to Texture).

I know the AO faceting can be nasty in xNormals, some solutions are the way you export your objects. Typically I try to avoid exporting an OBJ while the object has a turbosmooth stacked on it.

When i'm ready to export, I kill turbosmooth and convert to mesh smooth and adjust smoothing, then export so it preserves the full topology (turbosmooth can export out in obj with some wierd topology sometimes)

Sounds similar to the tessellation method mentioned earlier. Personally I'm using Maya, however some of that is definitely still applicable. Thanks alot!

Mr_Paris

Serp wrote: »

The best for me is Topogun. Really high quality. I prefer it to max, maya and xnormals. Although I haven't done an indepth analysis or anything.

I think I'm going to try rendering out an AO in several of the suggested programs and post of the results. Thanks

Mr_Paris

metalliandy wrote: »

I usually just add the LP model to both the LP and HP slots in xNormal and bake that

I may consider doing a quick smoothed mesh just so i can have something in that slot, not sure yet.

Mr_Paris

CheeseOnToast wrote: »

Xnormal with the method switched to "cosine" often minimises or eliminates low poly edges showing up in areas you wouldn't want them to, like the sides of a cylinder for example.

Excellent Tip!

SpeCter

There is an edit button you know, no need to use one post for each answer.

00Zero

this will sound stupid, but for low poly objects i just plug the low into both the low and high in xnormal. you might get some weirdness in some spots, but overall it works ok.

i can set up mental ray material and change all these settings in maya but thats such a hassle, too much time.

elitewolverine

have you tried doing AO bakes in blender3d?

i personally enjoy the simple way it does stuff, and free

Snowfly

I was about to recommend Blender AO too, particularly Approximate AO. It's quicker than physical AO and noiseless, which makes it a pretty nice fit in a low poly texture pipeline, although you do lose a bit of control.

http://www.bigbuckbunny.org/index.php/approximate-ambient-occlusion/

Basically, it's the AO technique that was developed for Dead Man's Chest:

http://features.cgsociety.org/story_custom.php?story_id=3889

I used it a lot on a low poly recreation of Shanghai Expo recently, was definitely nice to be able to chuck out these clean AO bakes quite fast, since I was doing a huge amount of assets in a short amount of time.

kodde

I'll second StefanH.
Tried FAOGEN, nice speed and good results. But lack of options to tweak the results. Doesn't seem to be developed further since some while.

I usually use xNormal or Mental Ray.