Hello all,
I'm starting my journey into shader programming and In many respects I just don't quite know where to begin. I've opened up some HLSL shader files and messed around (with some success) but I feel like I'm blindly poking around in the dark. Does anyone have some good resources for learning HLSL and or GLSL? I've read a lot about the matrices involved (world space/object space/ tangent space) and now I'm wanting to dive in deeper.
I'm familiar with rendering assets in passes and comping them together in photoshop, however I don't fully understand what math is being used to put it all together. For instance, I know I need to use the "screen" blend in PS mode for reflections... but what is the screen mode doing and should I be mimicking that in a shader?
I know light is additive so I need to add the diffuse pass and the specular pass together with the ambient color so:
Diffuse pass = add to ambient/base
Spec pass = add to ambient/base
But what about the rest?
Reflection pass = ?
Shadow pass = ?
Glow/emissive pass = ?
Thanks in advance PCers!
Replies
Very broadly speaking, every "pass" you've mentioned tends to be either additive or multiplicative.
And really they're not separate passes in shader terms, as a multipass shader takes the previous output as input for it's own routines.
As far as photoshop math gors, I have a link for that on my home pc. I'll post it here later if someone doesn't beat me to it.
Oh man I can't wait for that tutorial Sounds great. Thank you!!!!
Yes please.
http://dunnbypaul.net/blends/
http://www.cg-academy.net/es_catalog/product_info.php?products_id=64
http://eat3d.com/shaders_intro
I made some very basic intro videos here:
http://www.cgbootcamp.com/tutorials/category/cgfx
cgfx and hlsl are essentially the same btw.
reflections are added, you will not be using 'screen mode' in any non-stylized shader.
I'm using a directionaly masked overlay blend to fake SSS in UDK on a current model.
I think it's a lot easier to think about how material properties work together if you think about them in a physical sense.
Reflection and Specular is light that bounces right of the surface.
Diffuse (and Emission) is light that's been absorbed, scattered inside and then partially reemitted from the object.
Transparency and Translucency is light that is absorbed, scattered and completely reemitted from the object.
So specular and reflection are technically the same thing, but for a shader programmer, the difference is that the specular is multiplied by the incoming light (nl*shadows), while the reflection isn't.
emissive is usually added to the ambient light, and thus isn't affected by lighting. the last line in my shaders usually look something like;
ambient * AO * albedo + reflection * SO + (albedo + specular) * nl * shadows
albedo is the diffuse map, so is specular occlusion, nl is the dot product between the light and the normal
Wow... so much good information here already and I have a feeling we're just getting started!
I never knew that screen mode was a hack in gamma space. That makes perfect sense because when compositing in shake or nuke with linear renders you don't use screen for reflections either.
Why does clamping limit my shading pipeline and what should be used instead? If I don't clamp(normalize) the dot product of my diffuse calculation then I'll end up with negative numbers as well. What exactly are you referring to when you say I should avoid clamping?
Edit: oops double post
Many of those photoshop blending modes only work correctly with inputs between 0-1, a shader may be carefully written to only have positive values but a screen operation will try to use the inverse of your input by doing a 1-x and give you completely unexpected results.
So how would you handle multiple lights in that situation?
Another question... When rendering passes I work as much as possible in a linear workflow inside of maya. When programming do I have to worry about linear vs gamma colorspace? How exactly does that come into play? Am I to assume that everything besides file textures are linear?
For example, you should really get a shader working with one light, ignoring gamma issues, before worrying about more complex problems.
I haven't started writing my own shader from scratch yet. I'm working on an expansion pack for a game that happens to need a graphical update. The graphics programmer at our company is booked solid working on another title. So we're having to make due with a programmer who's never done graphics before and myself. We're working together to make this game look the best we can. I felt I could be of more use if I learned as much as I could about shader programming.
That said, I'm not sure if I can post actual code so I'll just do some examples.
The code itself is very modular. The diffuse, specular and environment parts are all in their own functions and then called at the end via +=
so for example
final color += ambient (Diffuse texture map * ambient color)
final color += diffuse ( Diffuse texture map* dot(N,L))
final color += spec ( Diffuse texture map * specular equation)
* shadows
return final color;
I'm having trouble with the specular currently. I can't get it to look right. It's either blown out and too tight or too widespread across the surface. If I mult the spec by dot(nl) inside the specular equation it almost completely disappears because this is happening before it's being added with the color channel instead of after.
float4 Blinn( pixelIn input ) : COLOR
{
input.lightDir = normalize( input.lightDir );
input.viewDir = normalize( input.viewDir );
input.normal = normalize( input.normal );
float dotNormalLightDir = dot ( input.lightDir, input.normal );
float diffuse = saturate(dotNormalLightDir);
float3 halfVec = normalize(input.viewDir + input.lightDir);
float dotNormalHalf = dot(halfVec, input.normal);
float spec = pow(saturate(dotNormalHalf),25);
return ambientColor + diffuse * diffColor + spec * specColor ;
}
I forgot to add the diffuse texture in this one, you have to add that in, sorry for that, just saw that after thinking about that code :poly136:
Something like
If it's something like that then the exponent (specpower) is what will control the tightness of the highlight. Values of 1 or lower will be blown out, values over 15 get tight pretty quick.
If you want to just tone down the brightness, multiply it by something less than 1 before adding it to the diffuse. Also you'll run into blown out areas more often by using the diffuse as the color component to the specular, as anywhere in the hotspot is being effectively multiplied by 2 and taking it out of the 0-1 brightness range. This can be less of an issue if you have a HDR lighting engine, but if not you may want to use either dedicated spec color maps or the 1- the diffuse color if you're stuck with using the single map. That way you'll at least self limit to white (1,1,1) as the maximum possible value for the final pixel color after the add.
ambient = diffuseTexture * ambientColor
diffuse = diffuseTexture * dot(N,L) * Shadow
specular = specularTexture * dot(N,H) * step(0.0, dot(N,L) ) * Shadow
color = (ambient + diffuse + specular) * ambientOcclusion
color += reflections
you can put the calc for diffuse and specular components inside a loop that iterates over your lights
EDIT: forgot to step the dot(n,l) in the spec, otherwise you darken your spec too much, you just want to take it out of the unlit side, you could use smoothstep() instead to give a soft transition
www.bencloward.com/resources_shaders.shtml
After that I grabbed Luiz Kruel's shader production DVD from Eat3D, which in addition to being very well narrated, has a bunch of great examples, and takes you step by step from flat color to lambert to full-on current-gen shader:
eat3d.com/shaders_intro
For photoshop blend modes, Vailias' link has lots of good info on what and why. TAO also has a useful list of ready-to-use HLSL translations of the blend functions:
http://tech-artists.org/wiki/Blending_functions
AFAIK, there's a D3D function to do this (D3DSAMP_SRGBTEXTURE), but I haven't seen an equivalent in HLSL. Like kodde, I just use pow(C, gamma) --> pow(C, 1/gamma).
For specular power, you may want to consider clamping to eg. [0.05, 1.0]. Very low specular power values, at least from my observations, result in artifacts where the specular will spread but then cut off with a hard edge (basically past the light influence).
Wow that smooth step really did the trick! All the info you guys have posted has been very helpful. My boss is very happy as of this morning too since yesterday our team endured the Eye of Sauron for the game's lack of graphical updates Despite being an older game it's starting to look closer to current gen. Thanks everyone for all the help so far.
Unfortunately we can't do ambient occlusion real time and as much as I would love to bake some textures for it I think we're out of channels in our texture maps. I've tried baking the AO into the diffuse texture but it doesn't give as good of results. But at least it's something.
Vailias: I removed the diffuse color component from the specular and it helps a lot as well. thx
I've heard HLSL and GLSL are very similar... is this true?
Does anyone have any good books to recommend on the subject? We're working in HLSL for our current game but in the near future we plan to use GLSL for our next project.
Only drawback is that it cannot be pre-compiled. It must be stored as plain text through-out.
www.luxinia.de/index.php/Estrela/Shader (for my own editor I've built myself some shortcuts to offline compilers for cg, glsl or hlsl)
That way if I never need to add geometry shaders I can just add [Geometry].
A sample shader.
http://pastie.org/2872641
Afaik, it's not a huge optimisation, and the benefits of normalize(V+L) outweigh any minor costs.
and yes the approximation for it being an actual reflection vector breaks down after a while, but should be sufficient for angles up to 90 degrees from the light. In my experience this is used in the specular component only. So if the object you're viewing is between you and your lightsource, you wouldn't get spillover like that anyway, and you wouldn't use it at such a low exponent as to blow out over the whole front surface of the object.
A simple fresnel term isn't really light direction dependent.. so not sure why you'd use a half vector in the calculations anyway.
I guess i wasn't being clear when i mentioned fresnel. I use it to refer to the fresnel equations, or more specifically the fresnel component of the specular highlight. This is definitely light (and view!) dependent, but I'm aware that it sometimes also refers to simple edge brightening/fake rimlighting.
For reference, specular reflections do get brighter at glancing angles, and this is what the result might look like (increasing roughness from top to bottom, increasing halfvector from right to left). so it definitely makes sense to calculate fresnel with the half vector (since it's a measure of the angle between the light and the viewer).
in the absence of SSAO I've had pretty good success simply adding a precomputed map to the ambient channel - itll react to diffuse light that way which of course it won't if you bake into the diffuse and allows you to add colour rather than just the conventional black&white. Well worth a look if you can afford the extra map
For that a half vector probably wouldn't work as well just due to how its constructed, and the direction it points in space.
Also the reason you're getting differences between the two versions is the length of the combined vector will not always be 2, and just scaling the resulting vector to length 1 doesn't actually return a vector in between the two original vectors.
Also I forgot to mention: the reason for a half vector is specifically to compare to the normal of a surface, as its very similar to comparing the reflected camera vector to the light vector when looking for direct specular reflection.
Remember there is a big difference between a vector halfway between two others, and half the angle between two vectors. So for things like the Theta term in Schlick's reflectance model, a half vector will not generate proper output.
You can also use the regular c preprocessor for this. GLSL program is built from multiple strings, first string could be "#version 120", second string, "#define _VERTEX_", and then the program string, which separates sections trhough #ifdef _VERTEX_ #endif. Although it does look less nice as custom tags.
Aight so thanksgiving week has been crazy but now things are slowing down a bit and I can get back into shader stuff.
Since we're not 64 bit we don't have the ram to add another texture map for every ship. We're currently limited to diffuse, team color, spec, self illumination, reflection, and a bloom map for the full screen glow post process.
I figure I'm just going to have to live without ambient occlusion but maybe there's some whacky creative hack that could work. Thoughts?
Why don´t you just blend predefined colors into the diffuse for team color instead of using a whole map?
Elaborating on Specter's point, as long as you're not talking about fancy multiple colors per team, you can multiply a flat color on top of the diffuse, via a mask. This way your team color map will go from a full color RGB map to a single channel (ie, grayscale). Depending on your other maps, you can either stick this in an alpha channel, or combine other grayscale maps into a single RGB image.
For example, depending on how you're using your self illum and bloom masks, you could combine:
Team Color: RED
Self Illum: GREEN
Bloom Map: BLUE
Which effectively saves you two texture sampler slots.
Ops, should have been more clear. Our team color is a mask in the alpha of the diffuse. The second texture contains specular in the red channel, self illum in the green channel, reflection in the blue channel, and bloom masks are in the alpha of the second texture. We use dxt 5 for almost everything and dxt5nm for normal maps.
Memory is getting to be a concern with how many assets we have in the game. We would really love to breathe some new life into the graphics but we're going to have to be really careful about what we do since we don't want to cripple performance. It is just an expansion after all(for a 3 year old game).
The reason for a half vector is that a reflection vector breaks down on any surface that doesn't have an optically flat microstructure. The derivation for it is pretty boring (it's more or less a solution to a filtering issue). Leaving that can of worms aside, I guess your argument was that you shouldn't compare half vectors to anything but surface normals?
...Which isn't really accurate. Comparing the half vector and the light or view vector is exactly what you want. In the fresnel equations θ represents the angle between the light and surface direction (In a model applying microfacet theory like blinn etc, this is measured by H). Also in Schlick's own paper, he himself specifically uses H.V as input for his fresnel equation. I'm not sure what you think would be better.
I haven't considered baking the AO to vertex colors. I wonder if our game supports that...
I just had another idea though. If my normal map is DXT5nm format it's only using the red and the alpha channel so I should simply be able to manually recreate the swizzling that format does and save it as a regular DXT5. This leaves me with the green and blue channels free. Perhaps I could place a baked amb occlusion there?
I've downloaded nvidia's fx composer and am writing a shader from scratch in it. I'm quite enjoying the process so far. But now I've hit a bit of a road block. It seems that if I use:
dot(Light, Normal)
this will only give me an interpolation between white and black for my lighting. Thats great if you want to multiply it over the top of your diffuse map texture. But with this method the lightest pixel at any point on the model is the value of your hand painted texture.
So how do I create a shader that behaves similiar to in maya where if you increase the light intensity (with the light color set to white) the entire lit area becomes lighter than the diffuse texture value?
Another part of this that i'd like to look into is being able to create a stylized look by having the "dark" or unlit areas be a color of my choosing that instead of being multiplied over the diffuse texture would blend(interpolate) into the solid color.
So from my description I guess I'd need to split my diffuse shading into 2 parts? I.e. I start with my object (ambient color to show the mid range of diffuse texture), then interpolating to the the "dark" or unlit/solid color side of the object and then an additive process to brighten the object's lit side.
Thanks in advance all you shader wizards!
EDIT: I whipped this up real quick in PS to explain what I want to achieve. (I know the colors are ugly! but it's so you can see the effect lol)
Not exactly sure I understand what you want to achieve. But from what I understand you want the resulting color to be "blown out", i.e. go brighter than the diffuse-texture color right? Sounds like default behavior to me. I mean, the equation is diffuse light * diffuse texture. So by having a light source which is above 1.0 value you would get a resulting color which is lighter than the diffuse texture right?
For example, let's say the diffuse texture is 0.5,0.5,0.5 and the light is a bright white light of 1.5,1.5,1.5. That would result in the final color of 0.75,0.75,0.75 which is brighter than our diffuse texture.
Maybe you don't have thought about having the light final color going above 1.0? You could incorporate an "intensity" for the light which is multiplied by the light color?
For playing around with stylized look you are talking about I'd look into using ramp textures to define this. It's a nice and easy way to play around with it imo. I've only ever created this with node based shader editors so I'm not sure how to code it. But you should quite easily be able to find info on it. I know for instance that the TF2 shading whitepaper has some info on this, they use this technique for their shading.
hrm... so I guess inside nvidia fx composer it's just that my light color wont allow me to to above 1. I'll have to add an intensity multiplier like you suggested. Thanks!
However I'd still like to look into doing the diffuse calculation in an additive manner. Because my end goal is to create a shader that takes into account color temperature in regard to the light source.
diffuse texture * diffuse lighting gives you a result that is unnatural. I want to create a falloff effect based on the color of my light. As light gets less intense it shifts in hue and saturation as well as value. The brighter the light reflected, the less saturated the color, the darker the shadowed area is the more saturated the light color is. (my example ended up looking a little like skin but it's not supposed to be :P just a sphere painted cream color with a yellow light shining on it.)
My example isn't quite an exact look... it's more the concept that I would like to be able to write into my shader. I'm interested in creating a shader that will handle this dynamically based on the light. Offsetting the hue and saturation automatically based on the light color.
Thanks! So once I set up the ramp, does anyone know how to plug those values into the diffuse lighting calculation?
I haven't had time to look over the valve link. But I will for sure do that soon
Link to Xoliul Shader Think you want the core one.
That way, if the diffuse calculation gives a a light color, it will sample in one end of the ramp, and if it's a dark color it would sample in the other end of the ramp.
Sort of like "filtering" the diffuse results through a texture.