how do you apply AO to objects with tileable textures?

a-k-m · Apr 2012

Hello,

I have an object with tileable textures, which its texture coordinates are out of the 0-1 space. That means I can neither bake an ambient occlusion map nor a normal map.

At the moment I just threw the regular diffuse map into crazybump and baked out my normalmap there... but what can I do to have AO for the whole object, without having a map?

Hope someone can help me out.

Cheers, Andr

Ben Cloward · Apr 2012

How about baking the AO into the vertex colors? Or applying a second set of UV coordinates? The only other option I can think of is to do the AO in real-time in your game engine.

cptSwing · Apr 2012

you could bake simple AO into your vertices

Computron · Apr 2012

From the wiki:

AmbientOcclusionVertexColor?action=AttachFile&do=get&target=AOTuto.jpg

MrNinjutsu · Apr 2012

You could integrate it into the texture. Set the AO on top as Multiply for the blend mode. Probably best to change the hue of it too.

Kbrom12 · Apr 2012

I use a second UV channel to solve this problem

Computron · Apr 2012

Kbrom12 wrote: »

I use a second UV channel to solve this problem

Depends on your engine, but if this is allowed beware that it pretty much duplicates your verticies.

wirrexx · Apr 2012

is there any good tutorial on doing this in Maya? Cheers

marks · Apr 2012

Computron wrote: »

Depends on your engine, but if this is allowed beware that it pretty much duplicates your verticies.

It adds less than you think - does depend on the context though. Most AAA-level artwork has a lot of data saved in the verts already, adding another UV channel is comparatively not as much of a performance cost as you would think - in that scenario.

AlexLeighton · Apr 2012

Why not unwrap within the 0-1 box? You can still use a tiling diffuse texture, you'll just have to increase the tiling however much is necessary. Then bake your AO and multiply it onto the tiling diffuse in your material.

St.Sabath · Apr 2012

wirrexx wrote: »

is there any good tutorial on doing this in Maya? Cheers

very simple:
assuming youre using MR as a render engine,set your dropdown menu to "Rendering"(Yellow dots),then in the Lighting/Shading menu (Green dots)
select Batch bake(mental ray),go to the options of it.Next set your options in this dialog eitther to texture or verts and bake.you can play around with the different options here to suit your needs..
Then after you baked it it appears as ao only on the object,to mix it with the diffuse and other maps,go back to your Polygon menuset(Yellow dots) and then go to Color-->Color material channel,and then choose Ambient+Diffuse.

wirrexx · Apr 2012

St.Sabath wrote: »

very simple:
assuming youre using MR as a render engine,set your dropdown menu to "Rendering"(Yellow dots),then in the Lighting/Shading menu (Green dots)
select Batch bake(mental ray),go to the options of it.Next set your options in this dialog eitther to texture or verts and bake.you can play around with the different options here to suit your needs..
Then after you baked it it appears as ao only on the object,to mix it with the diffuse and other maps,go back to your Polygon menuset(Yellow dots) and then go to Color-->Color material channel,and then choose Ambient+Diffuse.

Much Appreciated!

Ace-Angel · Apr 2012

If you have direct access to the shader, overlay it ontop of your diffuse result based upon the shadow direction.

a-k-m · Apr 2012

First of all thanks for the advices!
I tried it with the radiosity. The results are not too satisfying. Also it seems that the 3point shader lite I am using is not displaying the vertex paint in the viewport.

Anyway, maybe I will try it with two UV-layouts. Do you guys have a quick explanation how I set up two UV-Layouts? Does the CryEngine support 2 UV-Layouts?

The solution of tiling the texture itself more would work if the uv-layout wasn't so widely spread.

Here's a pic of the object and its uv-layout:

Thank you for your help!

r_fletch_r · Apr 2012

Ace-Angel wrote: »

If you have direct access to the shader, overlay it ontop of your diffuse result based upon the shadow direction.

What do you mean by this Ace? AO isnt directional.

Ace-Angel · Apr 2012

I mean do what Uncharted did ;P

r_fletch_r · Apr 2012

Ace-Angel wrote: »

I mean do what Uncharted did ;P

link?

cryrid · Apr 2012

What do you mean by this Ace? AO isnt directional.

Shouldn't it be though? I might be understanding the term wrong, or at least how it relates to the shadow direction, but ideally you shouldn't see AO where a direct light is hitting?

Xoliul · Apr 2012

My shader shows vertexcolors, you could try that.

Ace-Angel · Apr 2012

I think it's this one?

http://advances.realtimerendering.com/s2010/Hable-Uncharted2(SIGGRAPH%202010%20Advanced%20RealTime%20Rendering%20Course).pptx

Basically, what Cyrid said, the AO map should 'appear' in the shadow-ed part of mesh, while anything hitting with direct light should be washed out, true values, bleached, etc whatever you want to do, but shouldn't have hard shadows/ambient.

Is it 100% correct? Nope, but we're still ages away from having pixel level AO from the Normal map as opposed to just vertex/general shape, and other then Fight Club, I don't think I have seen any other game engine do proper AO in self-shadowed models.

Olli. · Apr 2012

cryrid wrote: »

Shouldn't it be though? I might be understanding the term wrong, or at least how it relates to the shadow direction, but ideally you shouldn't see AO where a direct light is hitting?

Having a light shining on AO doesnt actually remove the AO. The AO is still there, but you just cant see it because the directional light is lighting up the surface, and therefore removing the effect of the AO from that area.

Afaik AO and diffuse/specular lighting are done in seperate passes, so there is no way to remove AO from an area by using directional lights. You can only "hide" it.

edit: just adding on to that: If you do what has been suggested here in the thread and bake the AO in the vertex colors, then the diffuse color of the model will be darker, and it wont be able to be cancelled out anymore (since even though AO and directional lights are different things, they are both still lights).

unstoppablex · Apr 2012

so if you baked vertex AO, how would that carry over to game engines?

Computron · Apr 2012

What do you mean? If your importer supports them and any potential intermediate format has them enabled, then they should cary over perfectly.

unstoppablex · Apr 2012

Computron wrote: »

What do you mean? If your importer supports them and any potential intermediate format has them enabled, then they should cary over perfectly.

i tried to do import it from 3ds max to udk and the shading didn't stick

r_fletch_r · Apr 2012

cryrid wrote: »

Shouldn't it be though? I might be understanding the term wrong, or at least how it relates to the shadow direction, but ideally you shouldn't see AO where a direct light is hitting?

you mask your indirect light with it. then add that to your direct. Its in no way directional or related to your scene lights. Theres nothing physical about AO though its just a fake to help simulate GI, so i guess you can use it like you want. Also far be it from me to critique the guys at naughty dog. they know their shit (i cant read powerpoint docs, so i cant comment on what they're doing.)

sprunghunt · Apr 2012

unstoppablex wrote: »

i tried to do import it from 3ds max to udk and the shading didn't stick

If you import it as a FBX it should bring the vertex colours in. Make sure you have replace vertex colours turned on.

http://udn.epicgames.com/Three/FBXImporterUserGuide.html

Then you just have to setup a shader with the vertex colour multiplied over the top of the diffuse.

unstoppablex · Apr 2012

ahh i got it working! i've done vertex painting in udk but didn't know vertex color information could be transferred without a map! pretty neat. thanks

how do you apply AO to objects with tileable textures?

Replies