Cartoon "-Shading" in Doom3?

Herr W · May 2006

Has anybody tried out to create a Doom3 (player-) model using that cartoon "-shading" technique connor o'kane developed for Quake III? (http://www.cokane.com/)

I build a Quake III Arena mod using this style and would like to rebuild (or even better: convert;-) it to run on the Doom3 engine.

- Yes, just to draw more attention to it! ;-)

http://monkeykombat.planetquake.gamespy.com/

KDR_11k · May 2006

It's just the old envmap trick. Of course that'll work. If you wanted to do more you could modify the normalmap pixel program to round its light values so the entire world would be toonshaded in one go and react dynamically to lighting.

Edit: BTW, you'll need cubemaps this time.

Herr W · May 2006

Ha, ein Landsmann!

- Thanks for the quick answer! ... Which brings up new questions:

Are "envmap" and "normalmap" shaderstages, working within the game-engine? Connor O'kane's technique is pretty basic: You clone the model, make it a little bigger, put a black texture on it and flip the normals to the inside. I did that for all player models, weapons and mapobjects.

The map itself was build of func_groups with the "_celshader" key on them.

Did I get you right, and even the cartoon effect for the models can be generated by the game-engine (not "manually" in a 3D-program, as I did)?

KDR_11k · May 2006

Hm, no, I don't think the shells can be added through the material definition. You COULD use a pixelshader that compares depth buffer values and adds black outlines without adding geometry (the Nintendo DS can do that in hardware) but of course that means knowing how to write pixelshaders.

The other part is adding the envmap to give the objects the proper shading instead of having them use gouraud shading or being plain fullbright, that could be done with modifying the normalmap shader, too. I don't think you'll need that for level geometry, though. That tends to stay in one place and you can manually paint the shading on that.

Toomas · May 2006

What would happen, if you forced anti aliaized pixels to be black, wouldnt it create some cell kind of effect?

KDR_11k · May 2006

I don't think the renderer knows about the antialiasing, it only sees increased resolution. Also it would require AA to be enabled in first place. You can get the effect by comparing each pixel's Zbuffer value with that of the surrounding ones, if the difference is above a certain threshold paint it black.

Herr W · May 2006

The syntax of the cartoon-shader I used looks like this:

models/players/affe/affe_text_01
{
{
map models/players/affe/affe_text_01.tga
rgbGen identity
}
{
map models/players/affe/shade.tga
blendfunc filter
rgbGen identity
tcGen environment
}
}
The first texture (containing the models main colors and a black square for the outlines) isn't effected by any lights in the scene.
The second stage creates a dynamic ("flat") shadow on the models' surfaces by throwing a half-white half-grey texture to it.

Would this (or something based on the same principal) work on the Doom3 engine?

I'm pretty stupid in those technical issues (- right, grafic-designer...) and would be glad if the whole thing could be kept as close as possible to what I already have.

- Thanks again!

KDR_11k · May 2006

My test shader looks like this:

<font class="small">Code:</font><hr /><pre>models/characters/player/kdr_hitomi/solid_toon
{
noselfshadow
diffusemap models/characters/player/kdr_hitomi/black.tga
{
blend add
map models/characters/player/kdr_hitomi/solid_d.tga
}
bumpmap addnormals(models/characters/player/kdr_hitomi/solid_local.tga,models/characters/player/kdr_hitomi/null.tga)
{
blend filter
texGen normal
cubeMap models/characters/player/kdr_hitomi/toon.tga
}
}</pre><hr />

The diffuse has to be black and the texture added over that in order to make it fullbright. The envmap has to be a cubemap (the toon.tga here doesn't actually exist, there are six textures with names like toon_pz.tga). If you want a black outline done via envmapping you'll need a camera oriented cubemap. I don't think you should use envmaps on level geometry, though. That can be hand painted.

For specifics, read the reference.

Herr W · May 2006

Cool! I think that should be most of what I need.

Is it just an example or does it belong to a model you build?

Herr W · May 2006

Hm... I guess, I was a little to euphoric last time:-(

Whatever I tried to get your material running failed. In fact I think I didn't understand the second part - the bumpmapping - and what textures have to be set in there.

Could you help me with that again?

At least I managed to put something together that isn't too far away from what I have in mind (i.e. close to what I build for Q3A):

http://monkeykombat.planetquake.gamespy.com/cartoon_scr_02.jpg

This is the syntax of the materials I created for it:

textures/bb2/outline
{
qer_editorimage textures/bb2/outline.tga
noShadows
noOverlay
sort decal
diffusemap textures/bb2/outline.tga
}

textures/bb2/gelb_01
{
qer_editorimage textures/bb2/gelb_01
sort close
noselfshadow
diffusemap textures/bb2/grau.tga
{
blend add
map textures/bb2/strand_gelb_01.tga
}
}

The cartoon outlines are created by sorting a (slightly bigger) black box behind a colored one. - Pretty basic and maybe not what should be used in the end.

As you can see from the screenshot, this method has two "bugs" in it:

Although I managed to get shades to the colored boxes (by using a grey texture as diffusemap instead of a black one), the effect is ruined by the "outline"-box which darkens the whole inside.

And secondly: Weapon impact is nearly invissible on that surfaces and I have no idea why. The floor material has teh same syntax and it worked there.

KDR_11k · May 2006

I think you can ignore the bumpmap for this, the envmap doesn't react to it anyway.

The "sort decal" bi may be a problem, after all that's what I'd expect the impact effects to have, too. Would "sort far" on the outline and no sort option on the box work? What happens if you spawn entities in there?

Herr W · May 2006

Thanks for that hint again! I just tried it out: If the outline is sorted "far" the colored entity has to be sorted at least "close" to be seen.

The effect seems to work with other entities. If they are inside a box, they are invissible, if they are infront of they aren't.

But anyway: My method is pretty unellegant, as i have to draw all entities twice. Besides that the outline runs along the border of all entities instead of deviding them.

So I'd still like to understand how your example material would work.

KDR_11k · May 2006

Oh, that just uses two-tone shading instead of the normal lighting. It doesn't add any outlines. I think a good way to do outlines would be simply to draw black lines on the edges of the texture and have those edges match the edges of your box.

Herr W · May 2006

Drawing the outlines to the textures would bring two major diasadvantages:
- The textures become GIGANTIC (or blurr at the edges),
- You see "out"lines withIN a surface, which might be acceptable for cubes, but I think would look bad on more complex shapes.

Do you have an idea how to keep the outer shapes from darkening the inner ones? Even if they have a "noShadows" key on them, they prevent the light from brightening the surfaces of the boxes inside.

Cartoon "-Shading" in Doom3?

Replies