Hey guys! long time no see!
During the quarantine I decided to get out of my confort zone and try doing some NPR, but keeping its usage closer to PBR and metal/roughness workflow. Using oren nayar diffuse model and GGX specular model.
This is where I got to!
This is a shaderFX shader (for it to work, maya must be on opengl (its the default) and the outlines work better with 16X AA)
The outlines were done using partial derivatives, thats why you need a large AA number, it still does not work between objects nor on 100% hard edges, but it solves some problems, as there is no need for additional geometry, its easier to compose, and there is no sorting problems.
I am porting it to OSL, but there are several limitations to OSL on Arnold, no proper derivatives nor custom closures to name a few, the solution is messy and inefficient so this shader I am not releasing, but if autodesk hears me (doubtful) and removes some of the limitations, I will release it too.
There are 3 vertex painting options, I will use the default color set names in the description bellow, but you can change it in the shader:
CEL: Use a default value of 1, then values bellow 1 pushes the cel shading away from the light, and above 1 towards. This is useful to have artistic control, IE on clothes, and for shadows in specific places, like in the char example bellow, under the nose and mouth.
OUTLINE: Use the red channel to mask outlines and the green one to modulate its width.
You can choose how many levels of CEL you want in the shader (0 to infinity), and even convert your HDRI to CEL using the "USE ENV STEPPING" option.
Still, its an artistic shader, so its a bit more complex than a PBR shader and it has more controls.
If you want for it to be the default shaderFX shader, just put the file on "Documents\maya\ShaderFX\Scenes"
You are free to use it commercially if you are an individual freelancer, or doing non-profit work.
Here are some examples!
Hope you like it and thanks for your time! Cheers!
And here is a test with the arnold shader, with complex lighting and shadows and indirect lighting: