Home Technical Talk

Xoliul's 3DS Max and Maya Viewport Shader

2456710

Replies

  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    Hey, I'm looking forward to the new version. This is great news!
  • airbrush
    Offline / Send Message
    airbrush polycounter lvl 13
    Xoliul wrote: »
    Okay I'm almost ready to release a new version. It has some new features (mainly IBL). I still have to look into the Max 8 issue, I have no Max 8 myself so I will probably have to find that somewhere.

    Airbrush: I checked, you're probably just setting the global SI level to 100, which means everything is 100% lit. Set it to 0 if you use an SI map, since the global level is just added to the map.

    Looking forward to the new release.
    Global SI level works as you would expect....I had it set to 0 with a map applied, tried with a greyscale map, color map, painted in different areas to see if it was working...no luck. I'll give it a go again tomorrow to verify.
  • Christian Hjerpe
    Offline / Send Message
    Christian Hjerpe polycounter lvl 11
    Looking forward as well to the new version!!
  • airbrush
    Offline / Send Message
    airbrush polycounter lvl 13
    ...confirmed...self illum is working correctly.
    It was something with the max scene..not sure what. Exported obj, saved mat. Started with a new scene imported and loaded up the shader and it worked....
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Okay guys, update is uploaded. Version is now 1.3 since there are quite some changes. A lot more cubemap effects thanks to Brice for pointing out texCUBElod to me. Better Fresnel effects, Image Based Lighting with a seperate or shared reflection cubemap. Removed the gamma stuff since it was confusing. And probably more changes and fixes that I forgot about.

    get it here, together with my updated cubemaps:
    http://laurenscorijn.com/projects/viewportshader
  • funshark
    Offline / Send Message
    funshark polycounter lvl 16
    Hey cool!
    :)
    Trying that right now!

    edit :
    SI works strangely ( especially now; it didn't work before ) > when I change the Global SI Level value, it removes progressively the normal map
    o.O

    edit2 : Ok it works now....... :/
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    It's supposed to do that.... when using self-illumination, there is no lighting, so you can't see your normalmap. That global level is just there to slightly brighten your model if it's all too dark, or to go for completely unlit mode.

    I don't understand how so many people have problems with the SI map, it's the simplest piece of code in the shader pretty much ?
  • funshark
    Offline / Send Message
    funshark polycounter lvl 16
    xoliul > but now it just burn the colors :] and not erasing the NM

    Well, about the SI, it seems that it works differently inside xnormal for instance ( friend told me that, I didn't try myself ). Maybe people try to do the things like they used to ?
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Well, please do show me what you problem looks like ?
    It really does what I expect it to do when I use it: http://i39.tinypic.com/avinv9.jpg

    edit: I made a little guide on how to enable shadows in Max 2008 and 2009:
    http://i43.tinypic.com/radoqw.jpg
    It's included with the shader from now on.
  • hyrumark
    Offline / Send Message
    hyrumark polycounter lvl 12
    Maybe some problems are caused by people using ATI cards, which have never properly supported Shader Model 3.0, and this is acknowledged on the ShaderFX site.

    Shader works fine for me, great job!
  • funshark
    Offline / Send Message
    funshark polycounter lvl 16
    Xoliul > oh, SI works great now; just as I expected. I just said that some people may expect other result. :)
  • JostVice
    Offline / Send Message
    JostVice polycounter lvl 12
    Hey Xoliul, I work with low scale models for the game engine, I would like to work with some blur in the shadows but the smaller value I can use with your shader makes them at this zoom level like this:

    bapdhaabm.jpg

    Other than that, good work. :)
  • RickFX
    it works perfectly imho!

    wip12.jpg

    - Rick.
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    JostVice: I'm sorry, I never thought anybody would want to work at such a small scale, but I can imagine upscaling your stuff just for presentation is not something you want to do. I'll explain how to fix this:

    Open the shader in notepad. Look for this piece of code, it's near the top:
    float Shadow1Soft
    <
    string UIName = " Shadow1 Blur";
    string UIWidget = "FloatSpinner";
    float UIMin = 0.0f;
    float UIMax = 50.0f;
    > = {0.0f};
    and change it to:
    float Shadow1Soft
    <
    string UIName = " Shadow1 Blur";
    string UIWidget = "FloatSpinner";
    float UIMin = 0.0f;
    float UIMax = 50.0f;
    float UIStep = 0.01f;
    > = {0.0f};
    That changes the stepping of the UI spinner to a smaller value than the default 0.1 step. I'll put it in the next version!

    RickFX, looks cool. Is that stuff normalmapped ?
  • RickFX
    no that actually is the high poly version. check my thread for progress if you want, i saved your shader and my light setup in my max startup setup, so i wont ever use anything else anymore, lol.

    - Rick.
  • JostVice
    Offline / Send Message
    JostVice polycounter lvl 12
    Xoliul, thanks for the reply, but still it doesn't fix :(

    I'm using the value 0,001 that is the smaller I can type, and the shadows looks like the pic I posted above :(
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Then I'm afraid the only thing you can do, is upscale your scene. Maybe you could try setting its default value tosomething really small, it's the last number in the piece of code I quoted. Not much more I can advise you to do.
    You shouldn't expect super-high quality wonders from that setting anyway, it's a hack made by me to improve the shadow quality a little bit. As soon as Neil Hazzard from Autodesk lets the world know how we can use 2010's soft shadows, that setting will be in part obsolete.
  • JostVice
    Offline / Send Message
    JostVice polycounter lvl 12
    Yeah, I see :( Anyway, loving it so far! I might change to 3dmax 10, AO and soft shadows sounds good :) Thanks for the answers
  • animax
    Offline / Send Message
    animax polycounter lvl 15
    Hey Xoliul, I have just started using your shader. Awesome results.But i am stuck in using multiple lights in my scene.


    I have two omnis in my scene but the shader shows the output of only one.I do have both the lights assigned in the shader.
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    There's a spinner for Lights to use. Default is 1, increase it to 2 or 3.
    I think this setting might need some more or clearer explaining, quite a few people are missing this and asking me about it...
  • animax
    Offline / Send Message
    animax polycounter lvl 15
    Thanks for the reply.

    Just figured, Active lights option which was lost in between IBL Cubemap and Diffuse map options. I set it to two in my case and it worked. Why is active lights option is in there? Am i missing something?

    One more question, if i have two lights in my scene but i could increase the active lights to 3. How come?

    Thanx !
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Yes, I just figured it out too, that setting is lost beneath all the ambient light options. I will move it up so it should be more obvious.
    Active Lights option exists to make the shader use only1 or 2 lights. Since the light selecting lists MUST contain a light, you would get the default light applied 3 times without this option.
    And yes, you can change this option regardless of what or how many lights are in your scene, since a shader isn't so advanced it can query scene information from Max and limit your options based on them. The shader just supposes the user uses a bit of common sense when setting things up.
  • 3Drobbo
    Hey Xoliul, Great work on this shader! just want to say great work and im sure everyone really appreciates your efforts with his shader.

    I use it for everything these days!

    Good stuff mate!
  • glib
    Just wanted to chime in to say thanks for the great shader, tried it out yesterday and it worked great. And yes that semi-hidden active lights box caused me some issues at first too. Right at the top would be great!
  • Junkie_XL
    Offline / Send Message
    Junkie_XL polycounter lvl 14
    This has gone thru some change since I last played with this. max 2008 x64 didn't handle the shadows right but 2010 seems to be doing fine. Fun to play around with.

    I'm a bit confused on the IBL stuff. Ambient Cube Blur needs to be set to 8.0 for me until it looks presentable. Then 9.0 & 10.0 show no changes. I feel like I am shooting in the dark here on trying to figure how this is supposed to make everything look better. I can see a difference when using the cubemaps you provide...I guess I was expecting the lighting to effect the model marmoset style or something? Right now it just seems to be a way to brighten things...? I think I can see surface changes if I physically animate the model to rotate instead of just orbiting around the model...but that requires the ambient cube blur to be below 8.0, which then results in something strange looking...too glossy or something?

    And while Fresnel Mask Hardness is a nice touch, there is something about the way fresnel was handled in 1.0 I think I like better than 1.3 I can't put my finger on whether it is fresnel that acts differently or specular... The highlights inside the model seem to have much brighter tips. But if I bring down the rim color to a .5 or .25 gray the outer rim is then too dark.

    Well anyways this is a lot of fun, thanks for working on it. :-) Can we expect shadow support for all 3 lights in a scene in the near future?
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Thanks for the kind words, guys.

    Junkie: great to read this sort of feedback!
    Shadows shouldn't be too application dependant, although 2008 64bit crashed on them in my case (or invalid shader), but 32bit was fine. Keep in mind 2010 has a bug where it doesn't remember the lights set in a shader when you reopen your file. Autodesk is to blame here.

    IBL: yes, it's normal that you have to blur really hard, that way you can use the same file for reflection and ambient. The blur setting actually cycles through MipMaps and you want it to go through the lowest one possible almost (2x2 px). This is also the reason 9 and 10 don't do anything: mipmapping has hit the bottom limit and can't resize it any smaller.
    If you want to figure out the IBL a bit more, use a white model with considerable surface detail (a baked lowpoly or a midpoly object work best), and set the lights to 0. That way you can just fiddle with pure ambient settings, and later on add the lights into it. It's normal that a shader like this requires considerable tweaking to get it right.
    It does show that this feature might need some documenting/ explaining. I haven't really used it myself outside of testing while coding, will have to do that sometime.

    Fresnel mask: there's actually two methods in there as of 1.3. The old one masked the fresnel based on where the reflection vector was pointing: if the reflection traced to a point beneath the horizon, the reflection was masked (can optionally be turned off). This looked really good for glossy reflection type stuff, but didn't really work so well for more dull rimlights. So now the second method masks based on where the surface normal points (the more downwards, the more masked, hardness controls this ratio), which is like the TF2 rimlight (setting it to 0 equals no special masking). It's useable on almost everything. Keep in mind: this rim is masked by your specmap if you use one, you have to use a reflectmap (plain white for example) if you want different behavior. Also, I don't see why you would use gray? Just change the multiplier value to change the strength, it defaults to 2,0 but can go to 0,25.

    3 Shadows: well, hum, I explictly decided not to. Main reason it is Really, really heavy. Loading it can take up to a minute on some slow computers if it has 3 shadows, and performancewise Shader Model 3 can't even handle 3 blurred shadows (I hit the hardware limitations), so it would be 1 soft and 2 hard shadows. If you really want this, I have an old hacky version of Doylle's shader, which has this. But in general I don't think you need so many shadows in Max, other applications/game-engines are better with shadows anyway.
  • Junkie_XL
    Offline / Send Message
    Junkie_XL polycounter lvl 14
    Yep I seen I had to re-apply my omni's even though it says they are assigned upon file boot up. No problem.

    Interesting info on SM3, wasn't aware it could only handle one soft shadow. Can you support SM4 or are you limited to whatever max supports in the display driver? Isn't max 2010 supporting multiple soft shadows when turned on? (I noticed I have to keep that set to hard shadows for the soft shadow control in your shader to work). Even though yours only has 1 soft shadow I do like the control I get with it a lot more. Max 2010 seems to have lost shadow opacity and doesn't have anywhere near the features your shader does. Shadow opacity was moved to the real-time AO exclusively it seems?

    The one thing I found amazing was that my 7900GTX really slows down and chops on the real-time AO in 2010...but on my GTX280 at home it runs it handles it like a champ, no frame drop at all. I thought all this stuff required Dx10...people are really squeezing a lot of out Dx9 still...amazing. Anywho, maybe you could still support the multiple shadows in v1.4 and just mention that warning for slower comps? The GTX200 series really handles whatever you throw at it.

    Your explanation of IBL cycling mip maps makes perfect sense now. I wasn't sure what I was looking at before, heh. I'll have to mess around with reflection maps some more. Yeah a doc would be sweet.

    There really is a lot of tweaking involved with your shader but that is half the fun IMO. Concerning fresnel, I was comparing 1.0 & 1.3 side by side and to pull off the same effect 1.0 shows I had to tweak rim to .6 gray so the inside highlights weren't so white. Pulling down the multiplier to anything lower than 2.0 destroys the nice outside rim. There is a different sort of balance with the highlights. It's all good. 1.3 is starting to grow on me and I think I like it more now. :-)

    Just using my diffuse in the specular slot atm. I'll have to see what happens when I make a more accurate spec.

    Thanks again. Fun stuff. :-)

    edit: one more thing, you might want to leave "flip green" checked on by default. I've never used xNormal for baking so maybe this prefers it to be off...I just know I always have to flip it whether it is with your shader or Ben Clowards, etc. I bake with max.
  • rasmus
    Hey, a bit late to the party - thanks for a great shader!

    I have a problem though - it seems I'm not getting any filtering on any of the cubemap/reflection maps, which creates ugly results like below. I've double-checked everything to do with mips in both Max and driver settings, but to no avail. Am I wrong in assuming they should filter? I can set the blurring to maximum, but then that's kinda the same as having just a flat ambient color... This is with blurring set to 6, where filtering would be nice :)

    Also, is there any way to control whether or not the other maps use mip maps, or is this forced somehow? Compared to the standard Max DX display textures easily get very fuzzy...

    xoliul_issue.jpg
  • Brice Vandemoortele
    Offline / Send Message
    Brice Vandemoortele polycounter lvl 19
    Regular cubemaps have their mipmaps generated 'per face', resulting in different colors and 'seams'. You should use cubemapgen to have properly prefiltered mipmaps that blurs accross the edges of the cube.
  • rasmus
    I thought the cubemaps included with the shader was pre-filtered properly?

    Also wondering about the mip-maps for ordinary textures, right now things get pretty slaughtered... Is there a way to get anisotropic filtering? Or turn mips off completely?
  • t4paN
    Offline / Send Message
    t4paN polycounter lvl 10
    Valandar wrote: »
    But where do I put the shader files itself in MAX's file structure?

    Personally, I've got a separate folder called, well, "shaders" on my desktop and get them from there, kinda like any texture file I'm using. I don't use the default shader folder because I don't want to be searching through the folder to find the one I need.
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    rasmus wrote: »
    I thought the cubemaps included with the shader was pre-filtered properly?

    Also wondering about the mip-maps for ordinary textures, right now things get pretty slaughtered... Is there a way to get anisotropic filtering? Or turn mips off completely?

    Hi guys, had a bit of time off so not all that responsive.

    You're right rasmus, they should be pre-filtered, but only as of v1.3. If you downloaded them before I released v1.3, then you have the old version (you can tell by the rar including saparate IBL maps, now there's just 3 bigger files).
    If that still fails then I don't really know what's wrong: it works here with me and with other people.

    Regarding Max's mip-mapping. I agree it's bad, but it's still okay, I think UT3 has even worse mipmapping? I don't really think there's a way to fix this, Max only supports Linear and Point filtering options, Anisotropic just gets ignored and defaults to Linear. I could try experimenting with it, see if I can do anything, but I guess this is really application-dependant.
    Maybe maybe maybe, it helps to pre-generate you mipchains when exporting to DDS, and using one of those pre-sharpening filters on them that the Nvidia exporter has. It's just like pre-generating those cube mipchains actually is better than having the app do it for you at runtime. I have not tested this, so don't take my word, but it might be a fix.
  • rasmus
    Thanks for the reply Laurens - I did get the latest stuff, so it might be something related to my comp. Odd though that every single texture here has filtering, EXCEPT the cubemaps for your shader :P I'm sure that's all it would take for it to look the way it should.

    Regarding mip maps, this is the first time I've ever seen any in Max. And there's no way of telling it not to use any at all? Mip maps really make me sad :)
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Rasmus, another thing you can do against mipmapping, is overriding texture filtering options in your driver settings. On my work pc here I have 8x anisotropic filtering forced globally, which seems to affect mipmapping also, since I can't even see any horrible mipmapping with my shader here.

    Your cubemap problem, really no clue. I must say, the special function I use to force mipmaps down for the blurring, required some tricky input that didn't make any sense (if you would understand: I couldn't pass coordinates to it as a single vector4, but had to pass them as separate X,Y,Z,W values or I got very, very strange results). Makes me think this function might misbehave on some hardware/ with some drivers. What graphics card do you have ? I've so far used it extensively on a 280GTX and a 9800GTX, they should work fine.
  • rasmus
    Thanks again - I managed to overcome the mipmaps (global 8x anisotropic seemed to do it), but the cubemap filtering magic you pulled might be too much for my trusty old GeforceGo :P I'll try cubemapgen or something to make my own suitably blurry maps I guess. Thanks!
  • Wahlgren
    Offline / Send Message
    Wahlgren polycounter lvl 17
    Really like your shader mate. Had some problems with SI (didn´t respond when i did anything) but that was v 1.01. Newest one worked perfectly.


    LaptopRender.jpg
  • StefanH
    Offline / Send Message
    StefanH polycounter lvl 12
    just dropping by saying thx for this wonderful shader!

    Unfortunately i cant show my tests because of NDA but i will show something in the near future. Thanks a lot for sharing this. Really nice to use as well :)
  • Sandbag
    Offline / Send Message
    Sandbag polycounter lvl 18
    bringing this one back from the dead with a question.

    I recently switched to Windows 7 and max 2010 64bit, when applying this shader to more than 10 objects it immediately crashes.

    At first I was applying it to the entire selection of objects, so it would just crash immediately. Then I tried applying it to one object at a time and it made it up to 10 objects. Once I added the 11th object though it immediately crashed.

    Is there a set maximum number of objects that can use the shader at once?

    My system specs are windows 7 (build 7100), Core2 dual 2.66ghz, GeForce 8800GTS, 6gb ram. Love the shader though in general, great work with it!
  • Xoliul
    Offline / Send Message
    Xoliul polycounter lvl 14
    Hmm, I'd blame that on either W7 or Max 2010. I use Max 2010 32 bit on Vista, and my shader is on like ... 50 objects at the same time now.
    Max 2010 is a bit more flmisy with custom shaders (it forgets settings), and 64bit max versions have had more problems with shaders than 32bit versions in the past.

    Try to see if it works in Max 2009 and Max 2010 32bit. Also I suspect you had no problems before W7 ? So no reason to try it in a different OS.
  • slave_one
    Offline / Send Message
    slave_one polycounter lvl 18
    Awesome work on this Xoliul. Really cool to see what people do with it!

    I am messing around with it myself a bit, but have a problem with the IBL cubemaps. If I for example want to load the beach.dds that is provided, it says "Beach.dds is an invalid file type". It also says this when I try to load it in the regular cubemap slot. I just hope I am not doing anything really dumb. Using max 2009 and Xoliulshader_1.3.fx. If anyone can point me in the right direction, much appreciated. Hope to post some screenies soon : )
  • BradMyers82
    Offline / Send Message
    BradMyers82 interpolator
    slave_one: I had this problem before with max 2009. For me, it was an issue with polyboost. For some reason, polyboost is known to break the functionality of cube maps inside 3ds max. I would get the exact same error as you. My solution was to simply install the 32 bit version without polyboost, and inside of 32 bit max I could use cube maps again. Hope it helps.
  • slave_one
    Offline / Send Message
    slave_one polycounter lvl 18
    Hey Brad, thanks for the reply! Unfortunately .. I have max 2009 32-bit and don't use polyboost. At least gives me something to think about. I really haven't done much to modify / plugins to max. Some UV tools and that is basically it.
  • Sandbag
    Offline / Send Message
    Sandbag polycounter lvl 18
    I tried it in 2010 32 bit, but when I switched to windows 7 I lost all my previous copies of max (for work computer). In 2010 32bit it does the same thing, I'm not sure if it was at exactly 10 objects there, but it still did the immediate crash thing as soon as I got past at least 8 or 9.

    I thought I remembered applying it to more objects in max09 on XP, but I didn't have either of those things to test it with again.

    Also it's worth noting that when I suspected it was the textures (I hadn't used a 2048 with the shader before then) I decided to apply a blank version of the shader (no textures, default settings), which also crashed under the same 10+ object circumstances.

    My temp solution was just to attach all of the parts together (so it's definitely not a polycount issue).
  • Ged
    Offline / Send Message
    Ged interpolator
    some people seem to have bloom and antialiasing etc on thier pics is this a feature of max 2010? Im on 2009 thanks Xoliul Im loving the fresnel and SI stuff :)

    also my self shadowing on the objects seem to have stopped working when I opened up my old project the self shadowing on the object was just lots and lots of horizontal lines and nothing I can do seems to get them working again?, tried deleting my light and adding a new one and setting it all up again etc as soon as I select my light in "shadow term" it all shows up horrible
  • rasmus
    Ged: The bloom etc are Scene Effects in max - Go to the Tools panel, hit up "Scene Effect Loader", then load from /maps/fx/scene. Double-clicking on the effect in the list loads it into the currently selected material slot for parameter editing.
  • slave_one
    Offline / Send Message
    slave_one polycounter lvl 18
    Never knew this, .. Ged, thanks for asking and Rasmus, .. thanks for sharing!

    In addition, does anyone know a good place where I can read up on shader and shader use in general and maybe where things like this are mentioned? I always get bits and pieces of info here and there and some things through experimenting. Anyways, thanks again, this rocks!


    P.S.:Tips on what could be wrong when I try to load (IBL) cubemaps are still welcome :P
  • Ged
    Offline / Send Message
    Ged interpolator
    rasmus wrote: »
    Ged: The bloom etc are Scene Effects in max - Go to the Tools panel, hit up "Scene Effect Loader", then load from /maps/fx/scene. Double-clicking on the effect in the list loads it into the currently selected material slot for parameter editing.

    ah very cool thanks alot didnt know that was even there, I take it you enable hardware antialiasing on your 3dcard in order to get that in the viewport? also theres no sharpen post process fx, be nice to get that marmoset look huh?
  • slave_one
    Offline / Send Message
    slave_one polycounter lvl 18
    Maybe this can help someone else out as well.

    The problem I had with loading the cubemaps was solved by disabling shaderfx.
  • rasmus
    Ged: Never used antialiasing, but the default Scene FX are a bit useless imo... A lot of them blur the entire scene and are hard to control - I've just stuck to "Star" for hilite glow. Maybe there's other downloadable ones somewhere, too lazy to look :)
  • Arkhange
    Hi guys !

    A co-worker talks to me about this cool shader in Max viewport, but I got few problems when I want to use it in Max 2010 :
    - First, only the "NoShadows" version works correctly, the other do a "Loading Error"
    - I can see the result of the "NoShadows" version only after I've selected all the objects in my scene (When they are unselected they just don't seems to use the shader properly) T_T

    I've updated the drivers of my graphic card (ATI Radeon X1300/1550, who can also manage PixelShader) but it doesn't work much better
    I got Windows Vista (32 bit), is it possible that it comes from my exploitation system ?

    I'm so pissed off cause I want to try this shader for my Comicon entry :s
    Please help me T_T

    Thanks by advance
2456710
Sign In or Register to comment.