Home Technical Talk

Xoliul's 3DS Max and Maya Viewport Shader

1356710

Replies

  • ParoXum
    Offline / Send Message
    ParoXum polycounter lvl 9
    Maybe your GFX is too old to run a SM3.0 shader iirc.
  • Bal
    Offline / Send Message
    Bal polycounter lvl 17
    Arkhange, if you can't get it working on your Radeon, I used it on the machines we had at work (with the Quadros) and it worked fine (with Max 2009 anyways), so you could always try it there.
  • Arkhange
    Thanks Bal !
    I'll try to update the drivers on my Quadro at work as soon as possible, cause for now with Max 2010 it doesn't work nicely (I got the same problem with selectionned objects :s)
    I've talked with Funshark and he tolds me that maybe I should try with Max 2009 too...
    Wait and see ;)
  • funshark
    Offline / Send Message
    funshark polycounter lvl 16
    yeah the Quadros cards aren't the best choice to run directX shaders :S
    For instance, the shadow-map is not working at all with the latests drivers for some reasons.. and fps is disappointing :o

    It's really strange that Arkhange, with his gamer card, cannot get suitable results... Maybe Vista doesn't like directX 9- I don't know it very much :o
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    It does work fine in 2010 here, I've worked extensively in it with the shader the past weeks.
    I'm suspecting your Radeon is just too old, and to be honest, ATi has a reputation of doing all sorts of weird stuff with shaders in Max. It's been many times that the only conclusion for problems with my shader, was the use of an ATi card.
  • Arkhange
    Thanks for answer Xiolul !
    So, it's time to buy a new card and bye bye the old crappy ATI card...
    It's so sad that I can't used this real time shader for this comp' T_T
  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    Hey there, I started using the Xoliul shader again and its great! I especially like how the alpha channel supports all the different grey values.
    One question though, is there a way to make a plane two sided with this shader?
    At least on my computer it backface culls planes and I would like to have them 2 sided.

    BTW, I am using 3ds max 2010 64 bit, shadows and all the fancy stuff are working properly, Thanks in advance!
  • SpeCter
    Offline / Send Message
    SpeCter polycounter lvl 14
    I can´t use this shader the normal version just crashes my 3dsmax(2008 64 Bit) an the no_shadows version just makes my model black :(

    Maybe it´s a problem with ATI, because i think my HD4870 should be able to handle SM 3.0
  • G3L
    Offline / Send Message
    G3L polycounter lvl 9
    Is the download for this shader still available? I tried the link on page 3 for the most up to date one but it doesn't work anymore :(
  • G3L
    Offline / Send Message
    G3L polycounter lvl 9
    Oops!! Nevermind, tried the links on the first page and I downloaded the latest!
  • onionhead_o
    Offline / Send Message
    onionhead_o polycounter lvl 16
    where do i plug in the opacity map. or do i have to included as a alpha channel
  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    you have to save it as an alpha channel. I use .tga but any file format that supports alphas should work.
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Sorry I didn't check this for a while guys. I made a FAQ page just now:

    http://www.laurenscorijn.com/viewportshaderfaq

    Brad: You can actually change that yourself. open the shader, go all the way down to were the technique is defined, and change "cullmode=ccw" (or cw, don't remember) to "cullmode=none". Maybe there even is a technique for this already. I'll try to add it in next version. I'm gonna update this thing big-time in the coming months anyway, as I just got a job as a 3D teacher and want to use this in my classes, so spending time on the shader is justified :)

    SpeCter: Max 2008 64bit is horrible on my system too, it crashes as well. Again: Autodesk are the bad guys here with their finicky support. Try 2009 or 2010.

    G3L: website revamp means some URL's changed. I tried to keep them in the same directory however, seems like I failed ?

    onionhead_o: alpha channel indeed, like Brad said. I have to do it like this because there's a limit on how many textures a shader can use and I'm close to hitting it, so opacity in alpha saves one slot.
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Writing-spree!

    Wrote a small article on how you can go about setting up the shader in Max:

    Viewport Shader Setup
  • Unleashed
    Offline / Send Message
    Unleashed polycounter lvl 19
    is that all you need(ie it should show up in the viewport using just that information) or is there anything else?
    ive followed that info to the letter, and still dont have anything show up(lighting just looks wonky on the model, no textures/ or anything else shows) :(
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Did you follow the FAQ ? I wrote it for this purpose. My guess is it's going to boil down to shitty ATi hardware again...
  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    Xoliul: Thanks so much for the response!

    So I added another technique following your advice Xoliul that has no culling. Simply named, "SM3NoCulling". And it works like a charm. Simply change the technique and I can quickly turn on and off Backface Culling.

    So I have everything I want with this shader now, once again a big thanks to you Xoliul! I'm really excited about this now, because I finally have an all in one shader for 3ds max; while all the other shaders I have came across are missing at least one thing I need.

    Here is the shader with the added technique in case anyone doesn't want to mess with the code themselves (although it was ridiculously easy to make this change).

    http://www.bradm3d.com/images/Paintovers/Xoliulshader_1.3_NoCulling.zip
  • G3L
    Offline / Send Message
    G3L polycounter lvl 9
    Yeah, some of the links you posted prior weren't working anymore.

    Thanks for the awesome shader Xoliul! Can't wait for the next update!
  • FAT_CAP
    Offline / Send Message
    FAT_CAP polycounter lvl 18
    Anyone had this problem with the shader and know how to fix it?

    shader_problem.jpg

    This is on my new computer with a new card (ATI Radeon HD4650) I had the self shadowing working perfectly yesterday but after saving the scene and reloading it today, anytime I turn the shadowing on I get glitchy shadows like that. Restarting MAX and creating a new scene I get the same results.

    Has anybody had this problem and know how to fix it?
  • FAT_CAP
    Offline / Send Message
    FAT_CAP polycounter lvl 18
    OK - wierd. Sod's law this decided to start working again just as I posted that last post. I seem to have fixed this by scaling the object down to pretty small then back up again. Im assuming this is only a temporary fix but it will do for now while I work more on my object today.

    I can't get the self shadowing on at all in MAX 2010 but the viewport AO looks great and would really compliment this shader. Gah!
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Sam, I just mailed you about this, should provide some answers I think.
    I'll repeat it here for other people:

    Since it seems more of a generic 3D error, try updating your drivers to the latest version, especially on laptops these can lag behind many versions. I suspect this could be the main cause.
    See if older versions of my shader also have this problem.
    Make sure to turn of shadow blurring in the shader (set it to 0)
    Try a bit more to get it working in 2010. I can't say what you're forgetting, but there's a lot to be tweaked and changed in the viewport's Configure menu. Also, ONLY hard shadows work with my shader in 2010, soft ones just don't show up, so make sure to set the type correctly.
  • Mr.Bluesman
    Hi,

    I recently got introduced to you shader via game-artist.net. I've been looking for something like this for 3ds max for a while. Got sick of messing with unreal to show off game models due to it's un reliability and constant random crashing. We use something not so dis-similar at work but it's for Maya and I prefer to use Max at home so discovering this was a welcome find for personal projects. Just wanted to say thanks for providing this very useful tool to the community.

    One question - is it possible/have you considered making the shader more game environment friendly by adding the option to blend between say 3 textures using vertex colours RGB? Each diffuse could have it's own spec,normal etc?

    Also the abilty to be able to fade off a texture using vertex alpha would be awesome and very handy for decals/grime etc

    cheers and nice work

    bluesman
  • FAT_CAP
    Offline / Send Message
    FAT_CAP polycounter lvl 18
    Thanks for the reply Laurens!

    I manged to get the self shadowing working great in 2010 by changing the viewport shadows from soft to hard as you mentioned. Its all working great now - I'm a bit baffled as to why it was showing those wierd shadows but it is all working now without having to change drivers etc (I had the latest versions anyway).
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Hi,

    I recently got introduced to you shader via game-artist.net. I've been looking for something like this for 3ds max for a while. Got sick of messing with unreal to show off game models due to it's un reliability and constant random crashing. We use something not so dis-similar at work but it's for Maya and I prefer to use Max at home so discovering this was a welcome find for personal projects. Just wanted to say thanks for providing this very useful tool to the community.

    One question - is it possible/have you considered making the shader more game environment friendly by adding the option to blend between say 3 textures using vertex colours RGB? Each diffuse could have it's own spec,normal etc?

    Fat Cap: good to hear. i probably need to do some more explaining on the 2010 part of using the shader.

    Also the abilty to be able to fade off a texture using vertex alpha would be awesome and very handy for decals/grime etc

    cheers and nice work

    bluesman

    Hi Bluesman. thanks for the feedback, always nice to hear this.
    About your question: I'm afraid that's pretty much impossible. My shader is built on Shader Model 3, which supports a maximum of 16 texture samplers. Multiplying the number of textures by 3 would push them way out of this 16 samplers range. Also, using shadows takes up 2 or 3 sampler slots, so go figure how fast I hit this limit :)
    Same goes for Booleans btw, those switches for "use map", I hit the limit on those already and had to recode parts of the shader to work around this issue.
    The only option I see here, is creating a new specific blending shader that can do this, but will be more limited on options (cubemaps, both ambient and reflection, will have to be dropped, possibly gloss). The UI would also become an ugly beast IMO, with all those texturesets.
  • SpeCter
    Offline / Send Message
    SpeCter polycounter lvl 14
    Xoliul wrote: »
    Sorry I didn't check this for a while guys. I made a FAQ page just now:

    http://www.laurenscorijn.com/viewportshaderfaq

    Brad: You can actually change that yourself. open the shader, go all the way down to were the technique is defined, and change "cullmode=ccw" (or cw, don't remember) to "cullmode=none". Maybe there even is a technique for this already. I'll try to add it in next version. I'm gonna update this thing big-time in the coming months anyway, as I just got a job as a 3D teacher and want to use this in my classes, so spending time on the shader is justified :)

    SpeCter: Max 2008 64bit is horrible on my system too, it crashes as well. Again: Autodesk are the bad guys here with their finicky support. Try 2009 or 2010.

    G3L: website revamp means some URL's changed. I tried to keep them in the same directory however, seems like I failed ?

    onionhead_o: alpha channel indeed, like Brad said. I have to do it like this because there's a limit on how many textures a shader can use and I'm close to hitting it, so opacity in alpha saves one slot.

    Thanx for your answer, but i rather stay with Max 2008 i have 2009 and tried 2010, but i have to say that i don´t like the newer Versions of max:(
  • Unleashed
    Offline / Send Message
    Unleashed polycounter lvl 19
    thanks Xoliul, tried at work and it does work, turns out it is ati not playing ball with the shader. sucks, would expect a 4000 series card to support it
  • System
    Offline / Send Message
    System admin
    Thanks for the awesome shader Xoliul, this will make a big difference in displaying realtime graphics for portfolio :thumbup:

    ps: i was wondering if there is a solution to the no shadows in max version 9 and under, googled around and found nothing plug-in wise.
  • Michael Knubben
    Specter: That's entirely understandable, given how very much Autodesk changed in those. It's like it's a different program!

    I am lying!
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    SpeCter: strange, I found pretty much no difference between Max 2008 and 2009 after working with them both, except for the problem you have. I'd switch to 2009 if I were you.

    Unleashed: I'd expect a 4000 series to work, but even those don't seem safe. Try updating drivers, or using alternative drivers like DNa or Omega.

    GCMP: You're welcome dude. Unfortunately, there's no plugin for shadows or anything, Autodesk just wants you to upgrade. You do get SSAO and Anti Aliasing with Max 2010 :)
  • Christian Hjerpe
    Offline / Send Message
    Christian Hjerpe polycounter lvl 11
    Xoliul just a quick question, how does your shader work with 3dsmax 2010? Im going to switch and hope that your shader works fine.
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    If it has worked for you on previous Max version, I think it's safe to say it will work on 2010. Shader support and stability seems to improve with every new version.
  • ScoobyDoofus
    Offline / Send Message
    ScoobyDoofus polycounter lvl 20
    It works in 2010 but I needed to disable soft viewport shadows in max to get shadows on the Xoliul shader.

    I cant seem to figure out how to get the ambient IBL to work correctly. I'm probably just doing something wrong.
  • undoz
    I just wanted to thank you for the shader. It rocks!
  • capone
    Offline / Send Message
    capone polycounter lvl 18
    Argh going mad here, looks like a nice shader but can't get it to work with my Max 9 at all. Everything appears white, I checked out earlier posts where someone had the same problem and they told to get this,

    http://www.game-artist.net/forums/support-tech-discussion/5422-3dsmax-viewport-shader.html

    but that's even worse, as soon as I load it I get an error and no checkbox options at all.

    Any ideas?

    edit : I have a ATI Radeon X700 128MB card
  • ZacD
    Offline / Send Message
    ZacD ngon master
    I know this is a 3DS shader, but when I use the 1.1 no shadow version of the shader it has weird glitches that randomly change (if you check/uncheck boxes, but not directly related to one box, doesn't change when you rotate, but does sometimes when you move points).

    2l8b7tw.jpg
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    What program do you use, since you seem not to be using Max ? Might that also be the reason you're not using the latest version?

    Capone: I'm afraid you didn't read the requirements very well. I sepcifically mention this shader needs ATi X1000 or higher. X700 series don't support Shader Model 3.0, so it's entirely normal it doesn't work.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    ah k, I'm using XSI and got a 9800GT, I was using Version 1.01 I'ma try 1.3 and see if that works
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    ohhhh XSI, that explains. Well I don't officially support that. Yet :)
    I'm going to get a license for Softimage, Max and Maya from work, so that could be a motivation for me to add support for more programs.

    I'm surprised it does work at all, I had guys try it in XSI versions and all they got was errors. Are you using the older XSI versions, or the Autodesk renamed "Softimage" ? If anything, I'd only support the latest version. Supporting multiple versions of multiple programs would just cause too much headache and take too much effort from my side.
  • SpeCter
    Offline / Send Message
    SpeCter polycounter lvl 14
    Xoliul wrote: »
    SpeCter: strange, I found pretty much no difference between Max 2008 and 2009 after working with them both, except for the problem you have. I'd switch to 2009 if I were you.

    Unleashed: I'd expect a 4000 series to work, but even those don't seem safe. Try updating drivers, or using alternative drivers like DNa or Omega.

    GCMP: You're welcome dude. Unfortunately, there's no plugin for shadows or anything, Autodesk just wants you to upgrade. You do get SSAO and Anti Aliasing with Max 2010 :)

    The problem with Max 2009 for me was that it crashed very often on my Computer, especially because of the Viewcube. I had to disable it to get it running smoothly.

    Maybe it changed, i will give it a second try if you say it´s worth it.


    Edit: Changed to 2009 and Shader is running like a Charm, ViewCube Problems seem to be gone to yay :)
  • TychoVII
    Offline / Send Message
    TychoVII polycounter lvl 13
    Hi. I can't seem to get the cubemaps working. Got my 3 lights set up, got my maps working, but cubemaps don't show up. I tried the ones you included on your website, and I THINK my graphics card falls within your specifications. It's an ATI Radeon X1600.

    I saw what BradMeyers suggested but I don't even know what Polyboost is much less if it's installed or not. I'm running Max 2010, got any ideas?

    XolCube-Help.jpg
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    You did turn on Fresnel effects, right ? If it's off, none of the stuff beneath it will work.
  • TychoVII
    Offline / Send Message
    TychoVII polycounter lvl 13
    Whoops...

    Thanks! :D
  • Junkie_XL
    Offline / Send Message
    Junkie_XL polycounter lvl 14
    Xoliul wrote: »
    Autodesk just wants you to upgrade. You do get SSAO and Anti Aliasing with Max 2010 :)

    How is viewport AA enabled in 2010?

    edit: I'm aware of forcing it in the graphics card options...I'm just curious if there is a new max window?
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    It's in the driver options. Where you used to set texture sizes and "match bitmap" and so.
    But ehmm, I did notice it to be rather blurry and sub-par compared to decent AO forced through graphics drivers.
  • funshark
    Offline / Send Message
    funshark polycounter lvl 16
    Xoliul > I thought it was for the wireframe only (???)

    edit > Ok; it was really for antialiasing, good to know :)
  • Junkie_XL
    Offline / Send Message
    Junkie_XL polycounter lvl 14
    Xoliul wrote: »
    It's in the driver options. Where you used to set texture sizes and "match bitmap" and so.
    But ehmm, I did notice it to be rather blurry and sub-par compared to decent AO forced through graphics drivers.

    Thanks. Looked right past that.

    It doesn't seem to work with dense scenes. Although I'm testing on a 7900GTX here at the moment. Just throwing a few objects on the screen work. I think 2 samples looks better than 4.
  • System
    Offline / Send Message
    System admin
    Hi, is there a way to display decals using your shader?
  • butt_sahib
    Offline / Send Message
    butt_sahib polycounter lvl 11
    just dropped in to say youve done an excellent job lauren :) im a fan :) i showcase all my models in your shader. This or marmoset or brices shader hehe :D
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    GCMP: not really. A decal system is already complex to do in a game, so it's really hard in Max without application code supporting it. Unless you create planes with an opacity map in the diffuse alpha and place them in front of stuff. No special blendmodes then, that requires a special shader.

    Thanks butt-Sahib!
  • System
    Offline / Send Message
    System admin
    Thanks for the reply, might just render this one then instead.
1356710
Sign In or Register to comment.