Cel Shading

purehilarity

Hello! 

I had an effect that I'm wondering if it is possible or how difficult it may be.

I'd like two-tone cel shader that reacts to light, I know this is doable. I'd like to be able to plug in two texture maps, one for the light color, one for the shadow, if possible, instead of using a material for each object or just darkening the value for the shadows, instead use a particular color.

I'd also like it to have an outline that is two-tone as well. This is something I don't think I've seen done in a video game before. Here's an example of what I'm hoping to achieve...



Just wondering if anyone has seen this done with the two tones for the outline, or if you have any advice or examples.

skankerzero

Yes. This is a very straightforward shader that lerps between two textures with a non stepped ramp that is 50% black and 50% white.

As far as the shadowed outline goes, it's nothing new but most artists choose to make the outline full bright. Keep in mind that you'll have to reverse the shadow and highlight because the shading in the outline will be invert that of the model because of the flipped normals.

purehilarity

Thank you Skanker. I think I understand what you mean

skankerzero

Did you ever make any progress on this?

purehilarity

Hey dude, we decided not to go the cel shaded route for the project I am doing. Personally I can't build shaders in Unity, I'm just a Maya/Zbrush guy. Was going to have our Developer do it til we dropped cel. Your description makes sense to me though since I learned that cel is done by creating a duplicate that is expanded with flipped normals if I understand correctly, so it could just have another texture applied to that shell.

Eric Chadwick

Shader Forge has an outline mode built-in to the shader editor, worth a look if you go back to this. Shader Forge is really very easy to use, if you've ever used any node-based editors, like Hypershade.

skankerzero

Shader Forge's outline node is good for visualization and a very standard outline effect, but I wouldn't use it for final production.
SF's node suffers from:

• Broken outline based on uv seams. A simple fix by having the shader read from a secondary uv channel where the entire model is planar mapped, but the SF node doesn't read for other uv channels.
• Lack of varying lineweight. Using a black and white texture or vertex colors can easily change the variation of line weight on your model. SF does not support this.
• SF doesn't support per pixel color control on it's outline either. The outline will only ever be one color across the entire shader. This can easily be fixed by allowing an outline color texture.
• SF doesn't allow it's outline to take lighting or be full bright.

Your best bet is to have someone custom write an outline shader for you. 




It's really sad that people aren't putting as much effort to help advance tech for non-photo realistic rendering for packages like this.

purehilarity

"It's really sad that people aren't putting as much effort to help advance tech for non-photo realistic rendering for packages like this." True!

ngw

skankerzero said:

Shader Forge's outline node is good for visualization and a very standard outline effect, but I wouldn't use it for final production.
SF's node suffers from:

• Broken outline based on uv seams. A simple fix by having the shader read from a secondary uv channel where the entire model is planar mapped, but the SF node doesn't read for other uv channels.
• Lack of varying lineweight. Using a black and white texture or vertex colors can easily change the variation of line weight on your model. SF does not support this.
• SF doesn't support per pixel color control on it's outline either. The outline will only ever be one color across the entire shader. This can easily be fixed by allowing an outline color texture.
• SF doesn't allow it's outline to take lighting or be full bright.

Your best bet is to have someone custom write an outline shader for you. 




It's really sad that people aren't putting as much effort to help advance tech for non-photo realistic rendering for packages like this.

Some of this information isn't quite true, perhaps it was in an earlier version: 

UV seams don't (or at least I haven't seen them) cause breaks. Hard edges in the geometry do, however. So you might need to bevel in a few places where you wouldn't otherwise. If UV seams do cause breaks you ccould always move your "real UVs" to a second channel and let the shader know where to find them, then make UV0 a flat planar set as you suggest. 

Line weights can definitely be varied by vertex colour. Additionally you can modulate line weight by factors such as line direction or normal direction, which can give a very organic, responsive look to the line.

Line colour can be controlled via texture. UVs will be the same as those on the base geometry (you can attach a node to pick UV set to read from).

I haven't tried all light types on the line colour, but directional lights at least do work.   

skankerzero

ngw said:

skankerzero said:

Shader Forge's outline node is good for visualization and a very standard outline effect, but I wouldn't use it for final production.
SF's node suffers from:

• Broken outline based on uv seams. A simple fix by having the shader read from a secondary uv channel where the entire model is planar mapped, but the SF node doesn't read for other uv channels.
• Lack of varying lineweight. Using a black and white texture or vertex colors can easily change the variation of line weight on your model. SF does not support this.
• SF doesn't support per pixel color control on it's outline either. The outline will only ever be one color across the entire shader. This can easily be fixed by allowing an outline color texture.
• SF doesn't allow it's outline to take lighting or be full bright.

Your best bet is to have someone custom write an outline shader for you. 




It's really sad that people aren't putting as much effort to help advance tech for non-photo realistic rendering for packages like this.

Some of this information isn't quite true, perhaps it was in an earlier version: 

UV seams don't (or at least I haven't seen them) cause breaks. Hard edges in the geometry do, however. So you might need to bevel in a few places where you wouldn't otherwise. If UV seams do cause breaks you ccould always move your "real UVs" to a second channel and let the shader know where to find them, then make UV0 a flat planar set as you suggest. 

Line weights can definitely be varied by vertex colour. Additionally you can modulate line weight by factors such as line direction or normal direction, which can give a very organic, responsive look to the line.

Line colour can be controlled via texture. UVs will be the same as those on the base geometry (you can attach a node to pick UV set to read from).

I haven't tried all light types on the line colour, but directional lights at least do work.   

Re-reading my post, I know that some of that stuff has been fixed now.

As far as the uv seams go, yes you can planar map uv0, but I'd rather be able to define which uv channel instead of using 0. It's a consistency and standards thing.
You're most likely right on the breaking. Generally, with the models I've been making recently, the uvs are generally broken along the smoothing groups to make sharp edges. Although I remember doing some tests on a smoothed cube that would still break the outlines based on the uv seams.

You're correct on the line weight. The last set of models we made have varying line weight. I have to check on this though because I think we're doing it in a very round about way as opposed to just running the offset through the outline input.

What I do remember is the outline input in SF not being very friendly to customization. We did heavy tests with it at the beginning of the year and opted to just make our own outline shader.

And I honestly don't know what I meant by "SF doesn't allow it's outline to take lighting or be full bright."
That makes no sense. I think I left something out in that sentence.