Shader help!

Cheathem

hello poly counters.Im wanting to create a shader but i have no clue how to do so.there mite be a shader that already does what i need,if so then could someone please point me in the direction? Anyways,ill try to explain the best i can about the shader i need.

Basically,i want to be able to do the following: Have a diffuse map and a ambient occlusion map on a model.And have a light source (point light,directional,ect..) that effects the ambient occlusion map.i want the light to show up on the ambient occlusion map.that way if your AO map is on multiply,then when it displays it on your model,there will be no AO where the light source is.So the point of all this is to have a realistic AO setup.

For example,normally if you have an AO map on a house,the inside will probably be dark (considering the settings you baked the AO with) and if you put a light on the inside of the house,the light will barely light anything because of the AO map.But if the map was effected by the light.then even with an AO map on your house,putting a light in the inside would light the house realistically.

So,any help with this possible shader?
sorry for the complicated explanation.but thats the only way i can explain.

SpeCter

I understand what you want and most of it should be possible, but wouldn´t it be easier to use SSAO or something similar to get what you want?

You could make it like you want with 2 (or 3)passes one diffuse pass and the ao pass which gets the lighting calculation you want and then gets blended with the diffuse afterwards, but i´m not sure if it´s worth the effort(having many passes+ambient map for everything)

Computron

All the SSAO i've seen ignores lights.

SpeCter

Hmm...legit point there, not sure if it´s easy to include light in the SSAO pass, but i think there was something which did include indirect lighting so it should be pretty straightforward to brighten up the AO depending on light distance and strength.

Ace-Angel

CE3 does that correctly IIRC, UDK still uses old-school AO passes.

SpeCter

That´s HBAO which accounts for light if i´m not mistaken.

chronic

ambient occlusion is named such for a reason, you should only multiply the AO map (or ssao) by the ambient lighting for the most realistic look.

Computron

I don't think HBAO changes anything with regards to lights either, I just tried it in BF3 and it didn't react to light.

Cheathem

SpeCter, would it be more resource heavy to calculate the lights into the AO map then to use ssao or another method? the only reason i proposed this shader is bcause i thought it would be a really fast way to have believable lighting and AO.

Cheathem

Does anyone know if there a realtime way of calculating where the lights hitting the mesh and transferring that to the uvs?

Drew++

I guess you can just rebuild the lighting by doing a regular dot(N,L), then lerp between a value of 1.0, the AO texture map, and the alpha would be the dot(N,L) you built...

so basically
AOMap = tex2D(your AO texture, texCoord)
NdotL = dot(N,L)
newAO= lerp(1.0, AOMap, NdotL)

you would multiply your "newAO" onto the diffuse texture or whatever :P

Hope this is easy enough to read.

*edit: here's a pic.

Cheathem

whoa,i dont understand any of the technical talk but thats exactly what im looking for!Can you also have a diffuse map with that? and is it resource heavy? any way i could get that to work in 3ds max or blender or unity? sorry for all the questions,im just excited

edit: duh,you said you can multiply that with your diffuse map.so scratch my diffuse map question.

Drew++

Ah, sorry. I should've asked what you were doing this in. :P

Is this real time? If so, there is a node based shader editor for Unity, so I'm sure you could do it there. It would work on the same idea.
-There also is the ShaderFX editor for Max, same idea would work.
-Inside blender you can have custom shaders, you could ask someone on the blender forums to help you out with a custom OpenGL shader.

If this is offline, then I'm not sure I can help out inside Max or Blender

The only problem with this is the light would probably leak through to the other side of the model and mask it out where you don't want it to mask out... So the best option is to have some sort of Luminance texture(lighting, and shadows) and lerp/mix with that. Blender Game engine has a luminance texture I think.

Cheathem

Yes,real time.Im going to look into making this in max.Thank you a lot!

Cheathem

about the light leaking through,would this happen because the light isnt casting shadows? if so,then wouldnt the problem be solved simply by using shadow casting lights.

equil

this is really simple. you just multiply the ao map with the ambient lighting. no inverse dot products or linear algebra neccessary. i like to post pictures so here's an example in unity. here the diffuse map is just a solid color but using a texture instead obviously isn't a problem.

actually getting ambient light is the hard part. in the above example i'm using a diffuse cubemap, but the easiest way is to just use unity's built-in ambient light, you can access it using the keyword UNITY_LIGHTMODEL_AMBIENT.

can't really help you with making anything like this in max.

Drew++

equil wrote: »

this is really simple. you just multiply the ao map with the ambient lighting. no inverse dot products or linear algebra neccessary. i like to post pictures so here's an example in unity. here the diffuse map is just a solid color but using a texture instead obviously isn't a problem.
actually getting ambient light is the hard part. in the above example i'm using a diffuse cubemap, but the easiest way is to just use unity's built-in ambient light, you can access it using the keyword UNITY_LIGHTMODEL_AMBIENT.

can't really help you with making anything like this in max.

Ah, very good idea. I didn't know you could access ambient lighting like that... :poly124: Did you do that with a shader editor in Unity?

Computron

is there a way to access the ambient lighting inside of the UDK material editor?

Cheathem

Sorry for the long reply, completely got wrapped up in other stuff and forgot to reply.but thanks equil,that actualy really helps alot.Once me and my friend recreate your shader,ill post some examples for you all to look at if your interested.
thanks again guys!