vertex color

thinsoldier

Could someone please link me to somewhere I can get a difinitive explanation of what vertex coloring is and why I'd want to use it?

And if possible, a good tutorial on...um...whats it called....uhh....rendering in ?layers? for compositing purposes.

btw if u have any maya specific ones that would be great.

KDR_11k

Vertex colors are colors your vertices can have (well, duh). They're a way of adding color information directly onto the mesh without using or modifying textures. Lighting, for example, is a common usage of vertex colors because you'd likely reuse textures on the scene and cannot paint all the shading on the textures.

KMan

Most console games do alot of their world lighting this way. Since they don't have a huge memory budget to use for adding lightmap textures, they bake the lighting into the mesh.

Eric Chadwick

Here's a handy tutorial. Uses 3ds max and Unreal, but the same principles are probably at work in Maya.
http://udn.epicgames.com/Two/VertexBlendingTutorial

malcolm

You wouldn't ever want to use it, it is pure shit, on the other hand when working on games you are often forced to use it as your pipe is too crappy to support lightmaps or there is not enough texture ram because the other game areas are using too much.

CrazyButcher

I'd say the opposite, if you can get away achieving the effect with them, use them. because they are really very low cost. no reason to use maxed out features of an engine when there is cheaper ways of doing it.

danr

Agreed with crazybutcher

they also give you an immediate "in viewport" view of what your lighting will be like. If you're using something like Max VertexPaint, manually touching up your shading is a powerful, controllable and satisfying method

malcolm

I still hate them as I can't achieve the effect I want using them.

danr

something like Max Payne 2 is a good example of what you can do with careful vert colouring.

As long as you can tesselate your polys to a high degree (important), it's an excellent way of seating props against floors and walls and stopping that "floating" effect, and defining shape in surfaces that might otherwise appear flat (rucked-up carpets, missing bricks in a wall, etc).

Granted, it's very difficult to get proper cast shadows (where lightmaps excel), but for neutrally lit areas it can be spot on.

malcolm

I still personally disagree, I'm very familiar with vertex lighting and I think it is junk, natural environment or not. You would have to subdivide your flat areas so much to get any type of decent lighting that there would be no savings by that point because your environment would be so high poly. Careful vert colouring can achieve good results but the time you have to invest in getting those results is not realistic and making changes for the art director with cut in shadows is super lame. The sooner we can do away with vert colours for lighting environments the better. Don't take this the wrong way I'm not trying to start anything here I just find them frustrating at best and limiting to work with.



Here is a shot from the ps2, I did this environment with about 50% lightmaps and 50% vertex colours, this workflow is simply not realistic for any type of game where the objects and world elements are changing during development because any hand lighting you have done will be blown away when the instanced objects are replaced or the environment modeler decides to completely change a location, I have played Max Payne 2 as well, my friend works at Rockstar Vancouver actually, the lighting looks nice for the photoreal look they were trying to achieve, they have a lot of soft shadows though, are you sure they achieved these with vertex colours? I would think they would need really high subdivision to get those final gatherish shadows in all the corners.

FatAssasin

Vertex colors are actually a really good way to achieve a psuedo GI look, because they're very soft by nature. In fact, you can use Radiosity in Max to calculate a GI lighting pass and then bake that into the vert colors. But I agree that having to change hand painted vertex colors is a royal pain. That's why I try and use as many omnis as I can, and instances wherever possible, to set up my lighting. Makes it much easier to go in and make big changes. Then save the hand painted touch ups for alpha or even beta if you can get away with it.

Ryno

Baked lightmaps are just far superior when it comes to workflow, any way you slice it. Drop in a couple of lights, turn on radiosity, bake, done! You don't have to go in there manually slicing up your geometry or just sloppily subdividing the hell out of it just so that you get good vertex color bleed. I'm not saying that you can't do good looking work with vertex colors, but rather that it just takes a lot more setup and subdivision of your geometry. With lightmaps, this isn't a concern.

KMan

It just sucks that games like Resident Evil 4 use vertex lighting on their environments. It would look like total ass if they used lightmaps on everything.

Eric Chadwick

If you're fill-bound, the vertex color is very attractive, since it's only a vertex shader. But if you're only transform-bound (and have RAM to spare) then IMHO lightmaps are the best.

malcolm

KMan, I'm not sure what you're saying, you are saying that vertex colours suck but light maps look like ass? If you lightmap an entire environment it will look 10 times better than anything you can achieve with vertex colours as well your render will look exactly like your bake so you don't have to worry about hand painting or your shadows looking completely wrong, when we can lightmap 100% of an environment the visual quality will take a huge step forward, I'm just waiting for texture ram to go up, hopefully on the ps3

CrazyButcher

lightmaps are surely the best you can get currently in terms of lighting, because it happens offline it can be insanely detailed (radiosity, caustics...)
the downer is that they are static and not to be altered quickly/easily on runtime. though I am sure in a year or two the per pixel lighting will get even faster and move away the useage of lightmaps (at least on super capable hardware )
lightmaps will have a long life, just as vertex colors, because there always be some "low-end" where you cant burn all the tech gadgets that exist.