Homeโ€บ Technical Talk

Xoliul's 3DS Max and Maya Viewport Shader

1246710

Replies

  • Junkie_XL
    Offline / Send Message
    Junkie_XL polycounter lvl 14
    Is there a way to turn "2-sided" on?
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Junkie_XL: i described this to Bradmyers a few pages back.

    Also, I made a video about setting this thing up:

    http://cg.tutsplus.com/tutorials/3d-art/create-beautiful-realtime-renders-using-the-xoliul-viewport-shader-in-3ds-max/
  • Junkie_XL
    Offline / Send Message
    Junkie_XL polycounter lvl 14
    Xoliul wrote: ยป
    Junkie_XL: i described this to Bradmyers a few pages back.

    Also, I made a video about setting this thing up:

    http://cg.tutsplus.com/tutorials/3d-art/create-beautiful-realtime-renders-using-the-xoliul-viewport-shader-in-3ds-max/

    setting cullmode to none worked, thanks.

    You might want to mention how viewport anti-aliasing prevents the post_bloom.fx scene effect from working like it should. At least that was my experience.


    edit: quick question...is your shader 8-bit alpha compatible? Having a really difficult time with transparency. very pixelly and 1-bit looking.
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    You might've changed the technique. SM3Masked is 1 bit alpha, I believe.
  • Junkie_XL
    Offline / Send Message
    Junkie_XL polycounter lvl 14
    Xoliul wrote: ยป
    You might've changed the technique. SM3Masked is 1 bit alpha, I believe.

    ok so change the cullmode on just SM3Masked back to cw?
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    A technique is just a bunch of general rendering settings.
    AlphaTest is one bit alpha,AlphaBlend is translucency. Cullmode can be used for 2sided materials. So you can just define a custom technique if you want.
  • Junkie_XL
    Offline / Send Message
    Junkie_XL polycounter lvl 14
    Xoliul wrote: ยป
    A technique is just a bunch of general rendering settings.
    AlphaTest is one bit alpha,AlphaBlend is translucency. Cullmode can be used for 2sided materials. So you can just define a custom technique if you want.

    Hi Xoliul,

    oh ok SM3 and SM3Masked create two different categories in the drop down box.

    Regardless, I changed everything with AlphaBlend to true and AlphaTest to false. tried both. Still seeing the same 1-bit result. I did notice when it sees through to a second sheet behind then that portion becomes 8-bit.

    hairSample.jpg

    The sheet on the left is closer to the camera. The sheet on the right is farther away. The portion that sees thru the first sheet to the back becomes 8-bit where they intersect.

    These are my settings in the script...
    technique SM3
    {
    pass P0
    {
    ZEnable = true;
    ZWriteEnable = true;
    CullMode = none; //clockwise culling
    AlphaBlendEnable = true; //alphablend allows for full range, smooth opacity masking
    AlphaTestEnable = false;
    VertexShader = compile vs_3_0 vs_main( );
    PixelShader = compile ps_3_0 ps_main( );
    }
    pass P1
    {
    ZEnable = true;
    ZWriteEnable = true;
    CullMode = none; //clockwise culling
    AlphaBlendEnable = true; //alphablend allows for full range, smooth opacity masking
    AlphaTestEnable = false;
    SrcBlend = One; // additive blening of fresnel stuff
    DestBlend = One;
    VertexShader = compile vs_3_0 vs_main( );
    PixelShader = compile ps_3_0 ps_fresnel( );
    }

    }

    technique SM3Masked
    {
    pass P0
    {
    ZEnable = true;
    ZWriteEnable = true;
    CullMode = none; //clockwise culling
    AlphaBlendEnable = true;
    AlphaTestEnable = false; //alphatest is for hard, 1bit opacity masking. might work better in max with draw ordering.
    AlphaRef = 0.333; //Alpha Reference for masked testing
    VertexShader = compile vs_3_0 vs_main( );
    PixelShader = compile ps_3_0 ps_main( );
    }
    pass P1
    {
    ZEnable = true;
    ZWriteEnable = true;
    CullMode = none; //clockwise culling
    AlphaBlendEnable = true; //alphablend allows for full range, smooth opacity masking
    AlphaTestEnable = false;
    SrcBlend = One; // additive blening of fresnel stuff
    DestBlend = One;
    VertexShader = compile vs_3_0 vs_main( );
    PixelShader = compile ps_3_0 ps_fresnel( );
    }

    }
    What is AlphaRef?

    I really have no idea what I'm doing...lol
  • Junkie_XL
    Offline / Send Message
    Junkie_XL polycounter lvl 14
    Alright I think it is just a 2010 bug. Haven't tested with 2009. Under viewport configuration apparently enabling hardware shading turns alpha into 1-bit. if unchecked then 8-bit is fine. downside is no shadows of course.
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Try changing the Transparency Quality in 2010, it does some strange stuff as well, might fix it.
  • Junkie_XL
    Offline / Send Message
    Junkie_XL polycounter lvl 14
    Xoliul wrote: ยป
    Try changing the Transparency Quality in 2010, it does some strange stuff as well, might fix it.

    It seems with each problem solved a new one arises...heh.

    Changing transparency to none certainly put things back to 8-bit while hardware shading is enabled for shadows...but now real-time viewport AO seems messed up. Very grainy. No adjustments to AO help...it either looks worse or non-existent.

    I don't need the RT AO, just looked kinda cool. At any rate. Thanks for the transparency heads up. :)
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    That's Autodesk's coding for ya ;)
  • Junkie_XL
    Offline / Send Message
    Junkie_XL polycounter lvl 14
    Curious, what other scene effects out there are useful to a person besides bloom? I'd love a scene effect that sharpens instead. I really like various sharpening options in marmoset but I like the lighting control and other things with your shader much better. Unsharpen mask in PS doesn't really excite me.

    If I could somehow sharpen the image like that while still using your shader that would be tits. :thumbup:
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Actually it should be possible to do a Sharpen post effect, someone just has to write it. I know that in Doylle's old shader there was a sharpen effect applied to just the diffusemap, so doing on the entire scene is really very feasable.
    I'd like to do it, problem is it's gonna take some research for me till I'm able to code that. I have to completely pick apart the Bloom shader and the way it works with Max until i can write my own thing. But I really want to do that sometime soon.

    Howeverr, I started on version 2.0 of my shader today. It's gonna be a big change, but it will generally be a HUGE improvement in ease of use, and it'll be way easier for me to update and change. I'll also be able to support a lot of different max versions through one single, unified UI. If I pull this of I think it'll be the first public tool of this kind out there :)
  • Junkie_XL
    Offline / Send Message
    Junkie_XL polycounter lvl 14
    That's cool that you'd consider it. Could you try Shoehorning Doylle's (?) diffuse sharpening effect into your shader? That'd be a great start. Then later if you got that working maybe applying that same thing to other individual maps like specular would be sweet.

    Adjustable saturation, brightness and contrast would also be awesome in addition to the bloom. A Master adjuster for those as well as individual Red, Green, and Blue adjusting would probably allow for any mood to be desired...set complete within max.

    That'd make for the ultimate viewport shader in my mind. :thumbup:

    Looking forward to your 2.0 :)
  • SnarG
    In short this shader just has sex with my eyes and they love it. However, I'm a Max 9 user and I'm experiencing the problem of the lights illuminating incorrect faces (possibly inverse?).

    Xoliul, I've read in your FAQ that this is a known problem and you are on the path to fixing it, which is highly appreciated. I'm really hoping to be able to render some portfolio material with it.

    And on another note your tutorials on splines over at CG tuts were great; really learned a bunch of new tricks from them, so thanks for that too.
  • Hazardous
    Offline / Send Message
    Hazardous polycounter lvl 17
    Just a quick shoutout to Xoliul for providing this shader to the world man, its really a top top notch shader i hope to show some of my personal stuff with it soon!!
  • zxcman
    Offline / Send Message
    zxcman polycounter lvl 10
    can someone give me a couple of post effects?)
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    SnarG: development has sorta ceased for the moment, I have a lot of other commitments at the moment that need to be handled first. Also, I'm realy doubting if I'll still fix the Max 9 bug. That program is almost 4 years old after all. Also, I have to rewrite code that works perfectly fine in 2008-2010, with the risk of breaking things in those versions. The rewritten code would be less performant and messier in any case. I'd just advise you to upgrade.

    Hazard: awesome! I'm realy curious to see what you'll come up with, big fan of your work!

    zxcman: the only useable post-effects are included with Max. And Bloom and post-star are the only decent ones as far as I know. Writing post effects is something I haven't done before, and the method to get them working in Max isn't really straightforward. Also, they've deteriorated to a very broken state in Max 2010, as Autodesk seemingly completely ignores working on the post effect loader.
  • zxcman
    Offline / Send Message
    zxcman polycounter lvl 10
    is it any chance to assign xol shader to default max render?
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Completely impossible. Shaders use an entirely different system than software rendering. If it's not working fast enough: get a better graphics card. Geforce GTX200 series work perfectly with my shader.
  • zxcman
    Offline / Send Message
    zxcman polycounter lvl 10
    thx for your replies
    /another qustion how can i record turnaround from viewport?
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Just set up a rotating camera (I assume you know how), and then use Animation > Make Preview. It's quite handy for viewport animations actually.
  • zxcman
    Offline / Send Message
    zxcman polycounter lvl 10
    great!thx!
    i have ati radeon 3600.this video card "cheats" with shader and i cant use more than 1 light with your shader.do you know why?
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    You need to increase the number of lights used by the shader, it's at the bottom of all the light settings.
  • zxcman
    Offline / Send Message
    zxcman polycounter lvl 10
    i not so lame.i know the theory:)
    There is no resuslt of increasing number of lights.
  • SnarG
    Thanks for the reply Xoliul. Money is a limiting factor in the upgrading department I'm afraid so I'll have to find an alternate shader for the time being.
  • Hazardous
    Offline / Send Message
    Hazardous polycounter lvl 17
    I havent read all of this thread man, and youve probably been asked 1 million times - but ive fallen in love with your shader for max. Now im using maya and i dont have my xoliul shader *cry*

    Is the maya version on your todo list ?! I'd be willing to make a development donation!!
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Thing is, Maya really sucks compared to Max when it comes to shader support, the UI is really bad and the compiler doesn't give many error messages if it goes wrong. I could try, but it's not really high on the to do list, I have a lot of other things to do first and Maya isn't even installed on my PC...
  • garriola83
    Offline / Send Message
    garriola83 greentooth
    do cube maps work in this? i have a cube map and it seems to not show in the viewport. any ideas? my cube map is in .dds format
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    If the ones from my site aren't working, then you've got some weird bug. Another guy had it a week or two ago, i think it was a corrupt install, not sure, look for his thread.
    If my cubemaps do work, then yours is in the wrong format.
  • Quack!
    Offline / Send Message
    Quack! polycounter lvl 17
    It is because I am running 64bit 3dsmax that .dds doesn't work. They don't even load. Under 32bit they may load according to neox.

    Garriola are you even able to load the .dds?

    Just got back to the comp, will install 32bit max and see if it works.
  • garriola83
    Offline / Send Message
    garriola83 greentooth
    i dont see the cubemap at all in the viewport. it says its loaded. im also running 64-bit win7 3ds max 2010. and i also made my own cubemap so its not the one from xoliul's site.
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Do the ones from my site work ?
  • Flava-Fly
    Offline / Send Message
    Flava-Fly polycounter lvl 9
    I'm looking for some advice on getting a nice car paint effect to present my newest model in Max view port. Currently its all set up with Xoliul's fine shader, and I'm sure having seen his vehicle art the capability is in there somewhere but, I just cant bring out the best.

    The techniques I would usually use in environment prop work force me to have a kinda scratchy beat up high spec surface like I would use on an explosive barrel. I really want a cleaner more matted car paint texture.

    If someone could talk me through or screen shot some settings I would be massively grateful

    thank you!
  • garriola83
    Offline / Send Message
    garriola83 greentooth
    no i cant seem to produce the cubemap in the viewport. i put xoliul's cubemaps there in BOTH slots where it says cubemap but no cubemap appears. im using ver1.3 btw
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    I'm looking for some advice on getting a nice car paint effect to present my newest model in Max view port. Currently its all set up with Xoliul's fine shader, and I'm sure having seen his vehicle art the capability is in there somewhere but, I just cant bring out the best.

    The techniques I would usually use in environment prop work force me to have a kinda scratchy beat up high spec surface like I would use on an explosive barrel. I really want a cleaner more matted car paint texture.

    If someone could talk me through or screen shot some settings I would be massively grateful

    thank you!

    Do you have a picture of what you want to achieve ? Some stuff is more difficult but a lot is possible. Also the shader is built so that you can try out settings without even needing textures (well, you will need a cubemap for reflections). You can just try out settings like that.

    Garriola: try it in 32bit Max, like chaosquack says. If that doesn't help, mind telling me what graphics card you have? A really old one (geforce 6) or an ATi card might be the culprit.
  • Flava-Fly
    Offline / Send Message
    Flava-Fly polycounter lvl 9
    This is one of my reference pics it looks metallic and painted at the same time, I just cant find that balance!

    VW.jpg
  • roosterMAP
    Offline / Send Message
    roosterMAP polycounter lvl 14
    wow, thats cool. using from now on!
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Hey Flava, I made a quick example:
    VWpreview.jpg

    Files (max, textures, shaders):
    http://laurenscorijn.com/dump/example.rar

    White paint is always tricky to get reflections on. It tends to get overblown really easily. Especially when it combines with darker colors you need to have tweaked reflectionmaps to control things. i didn't really do that here, just went for some general settings, but they're not optimal.
  • Tom Ellis
    Hey Xoliul I got a quick question about your shader,

    I've been using it for ages and love it, but one thing I've just noticed, which for some reason hasn't appeared to be an issue up until now, is that the texture seems blurry from a distance. This may or may not be a Max setting, or possibly a setting within something other than your shader... either way, I'm not familiar with Max shaders enough to know!

    Basically, the detail in the texture or texture res seems to blur until you zoom up close when it becomes perfectly clear. Any way to solve this? I'm not at my home PC right now so I can't post a shot of it, but will do if you need it.

    Many thanks
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    That's just the MipMapping. Textures scale because of distance (to prevent moir
  • Tom Ellis
    Ah ok thanks for the quick reply. I had a feeling it was a Max thing.

    As you say, it's not an issue unless I start using details so small they would barely be noticeable in game anyway. In this case, it was some text on the side of a cardboard box for an env I'm working on, but even at distance it's still a damn sight better than in UE.
  • Flava-Fly
    Offline / Send Message
    Flava-Fly polycounter lvl 9
    Dude, you have gone above and beyond with this! thank you so much. Working a late one tonight but I'll try that out when I get back home. Nice model also ;)
  • Tom Ellis
    Ok I'm back with more questions... sorry.

    I'm working on a plant for the same scene I mentioned earlier, and I'm having issues with the culling.

    I think it might be linked to what was said a page or two back, but since I know absolutely nothing about shader language, I just saw a lot of code I didn't understand.

    I've enabled the two-sided thing by changing the culling line in the code, but now I'm getting this strange problem where parts of the mesh that are behind alpha'd planes, are being culled. It's difficult to describe but hopefully you can see what I mean.

    I'm using Max 2009 64.

    Here's two shots of the mesh I'm working with, first is in Marmoset, and second is using the Max shader.



    I know some people are thinking 'why not just use Marmoset if it looks ok' but I'm working on a hand-painted, semi-stylized env piece and the half-lambert shading and IBL in the Xoliul shader are making my other assets look awesome, so I'd like to stick with it!

    plantmarmoset.jpg

    plantxoliul.jpg
  • Flava-Fly
    Offline / Send Message
    Flava-Fly polycounter lvl 9
    Was your example made in Max2010? Wont open in my inferior '09 :< Might have to upgrade?! Could you screen grab settings for me? Sorry to ask.
  • Chai
    Offline / Send Message
    Chai polycounter lvl 18
    Does anyone know why Max blurs textures once I zoom out a bit ? (closeup zoom looks ok)
    I notice this problem in standard shader and xoliul, but poop shader (ben mathis) overrides the blur nicely.

    This has stopped me from using any shader besides poop's, and since that shader is so old now there's really only an option between xoliul and marmoset. (marmoset retains sharpness like most game engines)

    q_3dsmax_shaderblur.jpg
  • FAT_CAP
    Offline / Send Message
    FAT_CAP polycounter lvl 18
    Yeah I have noticed this with Xoliul's shader as well.

    I presumed it was just my mip-map settings in MAX but I have recently tried out a few new shaders at work and found that they keep the textures much sharper - not sure what voodoo they use as I'm not the one who writes the shaders but it would be great to get a tweak to try and replicate this in your shader Xoliul :)
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Hey that's new to me. I'll look into it, should be easy to duplicate it from Ben's shader to mine. Where do you get that, because I've never heard of it? is it the Agusturinn demo ?
  • Chai
    Offline / Send Message
    Chai polycounter lvl 18
    Hi Xoliul, ben posted the shader a few years ago .. I'll save you the time and upload it :
    http://www.arielchai.com/misc/poopinmymouth_uber_shader.fx
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    I just tested this in Max 2010 with Poop's shader, my old shader and a newer version with some edits (the only changes I can see that would make a difference here compared to poop's). I can not see any difference at all.

    Would you mind sending me that file where you do the comparision, Chai? It might be that my models and textures aren't optimal.

    Or try and see if this makes a difference for you: http://laurenscorijn.com/dump/Xoliulshader_1.4.fx
1246710
Sign In or Register to comment.