Maya Toon Outline issues.

_Connor

So, I have my base mesh on a MR render layer ( so i can use FG , and another render layer with a software renderer override, which contains my toon outlines.

I'm using offset mesh outlines because paintFX don't align to the profile properly. I have open edge and shader boundry outlines on.

The problem is the open edge outlines, which show fine in the viewport, don't render.

I've tried just using maya software renderer for everything, but then the outlines render over the top of my character.



The red circles indicate the problem I described, the green ones are another obvious issue that i have no idea how to fix.

edit:

This is the mesh toon outline creates to handle the profile.



It looks like a black texture with inverted normals.

Is it possible to mimmick this manually? say scale my mesh along it's normals and reverse them, then add a purely black texture?

MethodJ

Your last idea, yes that can potentially work. It simulates the outline look but it won't be perfect. And I bet MR wouldn't work well with that solution, even with the black shell set to not cast shadows. Since you're already rendering in layers it might not be a problem.

Two other ideas, all involve comping in something as an overlay:
1. Create renders using Viewport 2.0 (Maya Software 2). You could assign white materials to everything but your outline shell, so when you render you get black outlines on pure white, which you could then multiply on top of your MR render.

2. Render everything using Viewport 2.0, then do the white thing again, except using final gather and lights, you should end up with a greyscale image you can overlay on the original render.