Home Technical Talk

TF2 and TOON realtime viewport shader (3ds max)

polycounter
Offline / Send Message
rollin polycounter
(ok I´m creating a separate therad to make it easier to find the thread and to keep stuff together)

while I´m working on some custom models for tf2 i thought it would be faster to see the tf2-shader already in the max viewport..
this shader is more like a fake of the original tf2 shader but it´s already helping to get an idea what's going on

grab it if you want

thx again to Lumonix for shaderFX.. a very helpfull tool

(tested with max8 and max 2008 )










Team Fortress 2 shader v0.32

This shader is based on the real tf2-shader math.
That does not mean it's exactly the same but the visual result gets very close.

The Update to v0.32 brings you
- vertex color support and
- a fixed bug in the ramp shading

You can find the Download page - HERE -

+ picture of material rollout


comparison_032.jpg










Toon / Comic Shader v0.60

changes since v0.58
- completely reworked the code by hand going away from shaderFX. So I can clean up the code and I have more control over it.
- improved the sss-fake
- improved the rim lighting, it's now softer and easier to handle
- added cubemap blurring per value or per gloss map and masking through specular map
- you need a max version capable of displaying realtime hard shadows in the viewport (max 2009 or 2010, not sure)
- opacity can be read from diffuse color value
- allows up to 2 shadow casting lights


changes since v0.57
- fixed normal depth setting
- changed and fixed rimlight calculation. It's now working more the common way. You can offset the rimlight light source a bit (maybe have to increase the range) and exclude the normalmap from the rimlight calculation
- also added a simple sss fake. Have to get deeper into this but for the beginning it does a good job in fighting against black-skin-shadows

changes since v0.56
- zbrush material technique

changes since v0.55

- fixed colored specular map turned monochrome
- re arranged UI

changes since v0.53

- LitSphere support
- all major texture slots can be disabled/enabled
- normalmap red and green channel can be inverted
- added technique for 1 light/pass support - choose this if you're using the shader with the default scene light
- rim light can be calculated from normalmap or surface normals
- diffuse to ambient "leaking" can be influenced
- global opacity can be changed

changes since v0.51

- changed from shader model 2.0 to 3.0
- better rim light handling
- better ramp shading handling
- modify the nm blue channel
- some minor bugfixing



You can find the Download page - HERE -

+ picture of material rollout ( v0.58 )

toon_055_example.jpg

toon_053_example.jpg

toon_05_example_Ramps.jpg









the RampTextures

And here are some ramp textures for you to use DOWNLOAD


TF2 example:

ramp_tf2.jpg

the LitSpheres

Example LitSpheres DOWNLOAD

Example 4:

LitSphere_example_4.jpg

Replies

  • Bal
    Offline / Send Message
    Bal polycounter lvl 17
    Hmm, under max 2009, my model becomes invisible when I apply the shader on it, even with a normal map and ramp texture loaded, and lights in the scene.
    Will try messing around some more with it, I know you said you hadn't tested it in 2009.
  • rollin
    Offline / Send Message
    rollin polycounter
    I´ll grab the max2009demo and see whats going on

    additional.. if you have shaderFX you can get the project file, open it -> save 3dsmax shader

    you should get a shader that works with your version of max

    shaderFX Project File
  • JohnnyRaptor
    Offline / Send Message
    JohnnyRaptor polycounter lvl 15
    is this for 2010?
  • rollin
    Offline / Send Message
    rollin polycounter
    I´ve only tested it with max8 / max2008

    if someone has another max and shaderFX AND the shader is not working in this version of max, than it would be cool if he could provide me with the *.FX file from his shaderFX
  • Jason Young
    Offline / Send Message
    Jason Young polycounter lvl 14
    Works for me on 2010 64 bit(vista).
  • DEElekgolo
    Offline / Send Message
    DEElekgolo interpolator
    I don't thing you really need a ramp texture for this. If you want to get the unlit, cartoony style that tf2 has then look into a rendering method called Half lambert.
  • Bal
    Offline / Send Message
    Bal polycounter lvl 17
    Yeah, Xoliul's max shader does half lambert, haven't tried to see if you can easily recreate TF2 look with it.
  • rollin
    Offline / Send Message
    rollin polycounter
    JMYoung: thx for the info!!


    DEElekgolo: tf2 uses a fallof function to create the ramp-effect.. it´s not the "default" half-lambert..

    but why I´m using the ramp-texture here is bc I've modified my previous toonshader (which is based on a ramp-texture) to get a tf2 shader preview in max viewport fast

    this shader does only FAKE the tf2 shader behaviour.. but that's all I need while texturing :)
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Bal is right, you can get the TF2 look with my shader as well. It has Half Lambert, ambient cube, rimlight (3 kinds actually) and spec and gloss control with maps. There's just no ramped shading, but I light add it. It might be a little harder to achieve it in my shader just beacuse there are so many settings.
    But if you want Rollin, tudy my code to see how i did all these things, it's intended for people like you to learn from as well.
  • rollin
    Offline / Send Message
    rollin polycounter
    bc this question came up

    if you`re wondering what cubemap format the shader is using:

    take one of the maps that ship with 3ds max (you'll find them under ./max8/maps)

    or create your own one with CubemapGen from ATI


    you can create one out of single images like ben cloward describes here


    or simply convert a cubemap cross (3:4 format!!)
    you can find free ones here
  • Neox
    Offline / Send Message
    Neox veteran polycounter
    DEElekgolo wrote: »
    I don't thing you really need a ramp texture for this. If you want to get the unlit, cartoony style that tf2 has then look into a rendering method called Half lambert.

    you definitely need if you go for the TF2 look

    http://www.valvesoftware.com/publications/2007/NPAR07_IllustrativeRenderingInTeamFortress2.pdf

    the half lambert is a part of the shading, as is the ramp
    half lambert doesn't give you harder shading edges (like blurred cellshading) only makes the shading brighter and flatter
  • rollin
    Offline / Send Message
    rollin polycounter
    I would go even further and would say

    full lambert is a fall-of / ramp function
    half lambert is another one
    tf2-"lambert" is another one too

    I don´t know how they've implemented the ramp technically but theoretically I would say they've use simply a new fall-of function (tf2-"lambert") while ep1 just uses a normal half lambert fall-of/ramp
  • Neox
    Offline / Send Message
    Neox veteran polycounter
    they just plug the lambert in the uvs of the ramp, read the paper its all written there how they are using the ramp
  • CheeseOnToast
    Offline / Send Message
    CheeseOnToast greentooth
    I can't speak in techno-artist language, but having used the Source engine extensivly, I can tell you that you need a ramp texture (known as a lightwarp or phongwarp texture in Source) for sure. A smooth gradient from left to right that goes from black to 50% grey will give you identical lighting to not having the ramp at all. Anything above 50% grey will be overbright. The general-purpose ramp texture that is used on most things can be found in the pyro character textures folder if I remember right. They use a harsher, more stepped one for weapons. Even if you use a toon-shader style ramp texture, it's a good plan to blur the transitions a little otherwiswe the model tends to look too shiny/chrome-like rather than toony. Hope that makes sense.
  • rollin
    Offline / Send Message
    rollin polycounter
    ?! can´t realy find that point you're talking about (uvw-slot) ;p

    but thx for your kick in the ass to read the paper... :)

    should start to work on v0.3 now and make the shader a bit less fake#

    edit: and thx CheeseOnToast.. this helps indeed!
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Lol, you're coding the shader, but didn't read the paper until now :p ?
  • rollin
    Offline / Send Message
    rollin polycounter
    oh I´ve looked in it but more aimed on the single shader "layers" as on the technical mathematical base


    and there was no reason to do it..

    when I´m doing it now than just to try to REALY simulate the tf2 shader if possible

    not that it would be truly necessary
  • rollin
    Offline / Send Message
    rollin polycounter
  • rollin
    Offline / Send Message
    rollin polycounter
    ah sorry.. there was a little bug in the v0.3 download..

    should be fixed now

    the download link is the same but I´v updated the file
  • |*BILLY$CLINT*|
    Offline / Send Message
    |*BILLY$CLINT*| polycounter lvl 11
    Neox wrote: »
    you definitely need if you go for the TF2 look

    http://www.valvesoftware.com/publications/2007/NPAR07_IllustrativeRenderingInTeamFortress2.pdf

    the half lambert is a part of the shading, as is the ramp
    half lambert doesn't give you harder shading edges (like blurred cellshading) only makes the shading brighter and flatter

    Ah I was looking for this paper...Thanks a MILLION!!!
  • ThatDon
    Offline / Send Message
    ThatDon polycounter lvl 11
    Awesome improvement rollin, thanks for sharing! I'm hoping to put this badboy to use real soon :D
  • rollin
    Offline / Send Message
    rollin polycounter
    I've updated both shaders

    v0.31
    i´ve marked all settings which are no real tf2-shader values with an underscore
    the default values will get you a very close tf2-like result

    added spec, rim and exponent boost values

    added a texture slot to mask out the ambient cube



    v0.21
    added a texture slot to mask out the ambient cube
  • rollin
    Offline / Send Message
    rollin polycounter
    Update_08.09.jpg

    there is a new Update online

    - the shader v0.2x is now implemented in my toon shader (v0.51)
    - added vertex color support for both shader
    - fixed some bugs

    I've updated the first post
    check it out
  • rollin
    Offline / Send Message
    rollin polycounter
    Updated the first post!

    With: new update for the toon shader (now v0.53)

    changes since v0.51

    - changed from shader model 2.0 to 3.0
    - better rim light handling
    - better ramp shading handling
    - modify the nm blue channel
    - some minor bugfixing

    I hope the model-gets-invisible problem is now gone.. but I'm very happy for every positive or negative report !!


    toon_053_example.jpg
  • rollin
    Offline / Send Message
    rollin polycounter
    hey ho!

    I've finished version 0.55 which supports some important features and bugfixes.

    changes since v0.53

    - LitSphere support
    - all major texture slots can be disabled/enabled
    - normalmap red and green channel can be inverted
    - added technique for 1 light/pass support - choose this if you're using the shader with the default scene light
    - rim light can be calculated from normalmap or surface normals
    - diffuse to ambient "leaking" can be influenced
    - global opacity can be changed

    DOWNLOAD Page

    the new material rollout

    toon_055_example.jpg
  • rollin
    Offline / Send Message
    rollin polycounter
    small update to v0.56

    - fixed colored specular map turned monochrome

    - re arranged UI
  • hyrumark
    Offline / Send Message
    hyrumark polycounter lvl 12
    Thank you for sharing this, great shader!
  • onionhead_o
    Offline / Send Message
    onionhead_o polycounter lvl 16
    the alpha map function does not work for me. im using max2009. wut i mean by that is the mesh looks like theres no alpha map applied to it at all. i even checked the use opacity. and nothing happens.
  • kodde
    Offline / Send Message
    kodde polycounter lvl 18
    Your shaders seems sweet rollin. Damn shame that brand of 3D package hinders me from trying it out T_T
  • Hazardous
    Offline / Send Message
    Hazardous polycounter lvl 12
    Dude.... NICE! Cant wait to give it a pop!
  • rollin
    Offline / Send Message
    rollin polycounter
    the alpha map function does not work for me. im using max2009. wut i mean by that is the mesh looks like theres no alpha map applied to it at all. i even checked the use opacity. and nothing happens.

    I can't test the shader in max2009.

    But in max2008 it does work.

    So I'm bit sorry, but I can't guaranty it's working in all max version :/
  • onionhead_o
    Offline / Send Message
    onionhead_o polycounter lvl 16
    rollin wrote: »
    I can't test the shader in max2009.

    But in max2008 it does work.

    So I'm bit sorry, but I can't guaranty it's working in all max version :/

    no worries man. thanks for the reply tho
  • rollin
    Offline / Send Message
    rollin polycounter
    ok while working on my last model I did some work on my shader too.

    I'm releasing version 0.58 now while version 0.57 is used by GoMax

    new from v0.57 is a new technique that allows to see only the litsphere on the object. Nothing special.. just to get Zbrush materials into max viewport

    new with 0.58
    see image below
    - fixed normal depth setting
    - changed and fixed rimlight calculation. It's now working more the common way. You can offset the rimlight light source a bit (maybe have to increase the range) and exclude the normalmap from the rimlight calculation
    - also added a simple sss fake. Have to get deeper into this but for the beginning it does a good job in fighting against black-skin-shadows


    Download page is - HERE - again.



    toon_058.jpg
  • StickFigs
    Offline / Send Message
    StickFigs polycounter lvl 6
    In max 2010 the texture works when I render the scene but it looks wrong in the active viewport.
  • rollin
    Offline / Send Message
    rollin polycounter
    hi, are you referring to realtime rendering? bc I don't understand what you are comparing here. could you describe the problem a bit more or show a screenshot?
  • StickFigs
    Offline / Send Message
    StickFigs polycounter lvl 6
    rollin wrote: »
    hi, are you referring to realtime rendering? bc I don't understand what you are comparing here. could you describe the problem a bit more or show a screenshot?

    I thought this would allow me to see what the final project would look like in my 3d viewport window, basically making the 3d viewport window look like modelviewer.
  • rollin
    Offline / Send Message
    rollin polycounter
    so you're talking about the tf2 shader, do you? and you are comparing it with the hl2 sdk modelviewer?!


    if so, the default settings should already give you a quite similar result

    I don't know what you're doing.. if you use the toon shader to texture tf2 stuff it will not work (at least not with the default settings).. thats why there are two different shaders
  • rollin
    Offline / Send Message
    rollin polycounter
    I got my hand on max2010 and I really love the new functions for the viewport.

    Also tested the toon shader in it and for just a few minutes of playing around I really like the result

    f3b_full_low_04.jpg
  • Michael Knubben
    Rollin: sounds to me like he's talking about the realtime portion not working properly, which could be due to Max's bake? The fact it works in a render made me think that, anyway.

    Either way, Stickfigs, without images we can't help you.
  • Galago
    This is great, really helpful for texturing. On a semi-related note, is there some sort of way to duplicate the TF2 shading style when rendering?
  • rollin
    Offline / Send Message
    rollin polycounter
    could we pleaaaaase use the definitions "offline rendernig" and "realtime rendering" just rendering does say nothing


    you don't render an fx-shader offline.. you can use a script like this to make a high resolution screenshot though
  • Galago
    rollin wrote: »
    could we pleaaaaase use the definitions "offline rendernig" and "realtime rendering" just rendering does say nothing


    you don't render an fx-shader offline.. you can use a script like this to make a high resolution screenshot though

    Sorry, I'm rather new to this. Thanks.
  • Buck
    If only someone made it so you its not only real time but would render out same way too..
  • Charlottje
    Wow this is a great shader!
    I tried testing it with a car I modeled, but I can't seem to get the toon-y effect. If I set my rampcontrast to anything above 0, then everything just gets a lot darker.
    Anything from 0 to negative values, and everything gets saturated and a lot lighter. :/

    Am I doing something wrong?
    I made screenshots:


    Ramp Off
    Ramp On
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Hey Charlotte, it's because ramped shading requires a texture to work.
    If you're not afraid of the technical side behind it ;) :
    the shading calculation returns a value between 0 and 1. Instead of multiplying this to darken the unlit areas, this value is used as a UV-coordinate for a gradient ramp texture. Since a gradient only goes from left to right, there is no need for a height coordinate, you can just use a constant value for this. Once the gradient is sampled from the texture, this value is multiplied instead of the 0-1 of the original Phong calculation. Nifty huh?

    What you are seeing in your case, is 3DS Max returning black, because no gradient ramp texture is loaded. Just load one into the correct slot, I think the shader comes with a few of them.
  • rollin
    Offline / Send Message
    rollin polycounter
    heyho

    updated my main shader to 0.60

    changes since v0.58
    - completely reworked the code by hand going away from shaderFX. So I can clean up the code and I have more control over it.
    - improved the sss-fake
    - improved the rim lighting, it's now softer and easier to handle
    - added cubemap blurring per value or per gloss map and masking through specular map
    - you need a max version capable of displaying realtime hard shadows in the viewport (max 2009 or 2010, not sure)
    - opacity can be read from diffuse color value
    - allows up to 2 shadow casting lights



    DownLoad Page


    this here for example is done with the shader: unedited screengrab from max 2010
    toon_06.jpg
Sign In or Register to comment.