Anime Style Shading / Outline in 3DS Max & Maya

kaspar

Hi everyone,

I'm making some R&D for an animation project I'm working on and I want to give to it anime-look.

I've already read the whole discussion about GGXrd tecniques and I got how the tricks they use actually work and I want to replicate that tecnique whith some improvements that I think may work but there are a few things I don't know how to realize.

Does anyone knows if something like Blender Render Layers exists in Maya and/or Max ?
I need to give "priority" in renders to an object over another one and make it look "always in front of the other" even when it's actually behind.
For Max I've found someone mentioning "z-sorting" but I'm not sure it's what I'm looking for..

Here you can see how Blender "Render Layers" work:
http://blender.stackexchange.com/questions/18433/rendering-an-object-through-another



About shading I'd be glad to receive any advice about where to start for writing a custom shader that works also in the viewport for Maya or Max  (2017).

I made some research on my own but I've been able to find only out-dated links.

Thanks everyone for any kind of help!

Scruples

Check out the methods they used in Guilty Gear, they used Softimage but there isn't anything you can't do in 3ds or Maya too.

https://www.youtube.com/watch?v=yhGjCzxJV3E

http://www.ggxrd.com/Motomura_Junya_GuiltyGearXrd.pdf

While Maya and Max support "render layers". You'd still have to composite in an external program. Maya you'd apply a 'Background Material' to define a mask, in max it's called 'Matte/Shadow/Reflection'.

kodde

So are we talking real-time or offline rendering?

kaspar

Scruples said:

Check out the methods they used in Guilty Gear, they used Softimage but there isn't anything you can't do in 3ds or Maya too.
http://www.ggxrd.com/Motomura_Junya_GuiltyGearXrd.pdf

While Maya and Max support "render layers". You'd still have to composite in an external program. Maya you'd apply a 'Background Material' to define a mask, in max it's called 'Matte/Shadow/Reflection'.

As I wrote I already saw and read almost everything about GGXrd but thanks for the GDC conference official PDF link! (I missed that and I had just some low quality screens)

I'd rather think that it must be possible somehow both in Maya and Max even if maybe it will require some coding. 

kodde said:

So are we talking real-time or offline rendering?

I'd like to go for real-time shading/oulines like the GGXrd ones or at least anything that works also in the program viewport (for the same reasons: faster workflow and precise control of the shadows and outline during the animation / modeling process)

poopipe

you'll have to get very clever to create something that'll both render offline and appear representative in the viewports. 

That said, you can write both object based and post process shaders for max - a combination of the two should allow you to get a pretty decent result with minimal effort.  
It's been a  few versions since I did any custom shader/rendering  work in Max but you're likely to hit a few walls in terms of artefacting (for example max has historically used object creation order for alpha sorting ) its not really an issue if you're just previewing for game engine work but would be if you were wanting finished results. 

Maya is a lot more open - you can write your own viewport renderer and plug it in if the urge takes you. 

to be honest - I'd do it all in unreal these days