A lot of us have probably read the Vavle Dota 2 character and texture art guides by now. If you haven't, then GET ON IT!http://media.steampowered.com/apps/dota2/workshop/Dota2CharacterArtGuide.pdfhttp://media.steampowered.com/apps/dota2/workshop/Dota2CharacterTextureGuide.pdf
One of the techniques they talk about in the guides is creating a gradient from light to dark - light at the character's head, dark at their feet. It always bothered me that I couldn't figure out a way to bake out some sort of texture or mask that would do this work for me. So I finally got fed up and worked out a solution.
What I did, essentially, is bake lighting onto my low poly model with a light above my character's head. The nice thing about this technique is that you don't need to explode your geometry, or bake objects out separately, and it gives you a lot of flexibility.
THIS GUIDE IS FOR MAYA, BUT THE SAME TECHNIQUES ARE ALMOST CERTAINLY POSSIBLE IN ANY SOFTWARE.Step One - Set up your UV's
This goes for any kind of baking, but don't bake onto mirrored uv's! An easy way to get mirrored uv's out of the way is to simply offset them one full unit in the UV editor (so 0 to 1 becomes -1 to 0). When you bake, the software will ignore any uv's outside the 0 to 1 area.Step Two - Normals
The crux of this technique is that we are getting what we want by baking lighting from a single source, above the character's head. But if our light source is above the character then the bottom of their chin, under their arms, etc, will be completely in shadow, wont they? Yes they will - unless we change their normals to point straight upward.
Quick crash course on normals - a polygon's "normal" is what direction that polygon is "facing". So regardless of what shape or orientation a polygon is, if it's normal is pointing upward, then it will act like it's facing upward. Light from above will illuminate that polygon, while light from below will not affect it.
Usually the normals of your model will point outward, in a way that makes sense (top of your model gets lit from above, bottom gets lit from below.
But since we want all parts of our model to get lit from above, we want all our normals facing upward. To do this, select all of your geometry, go to the Normals Tab under the Polygons Shelf and select Set Vertex Normals. Hit the box next to it, actually, since we need to adjust the settings. Make sure your settings look like this.
X and Z values at 0, Y at 1 (pointing upward). Click Set Normal, and as you can see, the normals are all pointing upward now.Step Three - Light
We are going to bake lighting, so we need a light. Create a point light and place it above your character's head. Make sure you are previewing lighting, so that you can see the actual effect it will have (lighting > use all lights).
Okay, our problem here is that light has no fall-off, so all parts of the character are equally well lit. To fix that, go to the light properties and give it a linear falloff, then mess around with the brightness till the gradient looks like something you want. Don't worry if the contrast isn't intense enough, we can deal with that in photoshop. Here I set my intensity to 35. If your scene size is smaller, you will probably need a smaller intensity, and vice versa.Step Four - Baking Lighting
To bake lighting in Maya, you need to make sure the mental ray plugin is loaded (Window > settings/prefs > plugin manager > mayatomr.mll). Then go to the Rendering menu, and go to Lighting/Shading > Batch Bake (mental ray) and click on the options. Go to edit > reset settings.
-Use bake set override ON
-Color Mode to ONLY LIGHT
-Set your bake resolution, file type, anti-alisasing, and edge padding to whatever you want.
Hit Convert, and mental ray will bake out your texture. (check the script editor if you aren't sure where it saved it).
There you go, your very own mask texture!
Just play with the levels in photoshop to get the correct amount of contrast.
Hope you find it useful!
A subtle gradient from top to bottom: