Advice with Learning Shader Programming
polycounter lvl 10
I'm sure you have all read enough questions on this kind of thing (hell, I've read enough of them trying to find answers) but I just wanted some extra input on the matter.
I am interested in learning hlsl for shader programming, but I am struggling to understand where I need to start. I am currently getting used to programming learning javascript though Codecademy, but I am mainly just doing it to get used to an object oriented language. What I really want to do is get stuck into HLSL. I have seen a lot of places saying "just look at existing shaders and have a tinker" but without knowing the basic syntax of the language, it is hard to know where to start.
Should I carry on learning JavaScript to finish the course and get to grips with things, or should I move more on to a C based language straight away? As from what I gather, to understand hlsl I really need to learn C/C++. But I don't know what book or tutorials to go for to learn enough without wasting my time knowing the ins and outs of the language if it has nothing to do with shaders, or in fact do I need to learn the whole of C++ to learn how to write good shaders?
I have been looking at getting Accelerated C++ to learn it. Also the Microsoft DirectX 9 programmable graphics pipeline, as this sounds like it would teach me more on the Mathematics behind it. Can anyone recommend these books or suggest others.
I am fortunate enough to currently be working as an artist at a studio, so I don't need to learn these things to get me a job, I just want more knowledge on the matter to hopefully move into a more technical artist position
I am interested in learning hlsl for shader programming, but I am struggling to understand where I need to start. I am currently getting used to programming learning javascript though Codecademy, but I am mainly just doing it to get used to an object oriented language. What I really want to do is get stuck into HLSL. I have seen a lot of places saying "just look at existing shaders and have a tinker" but without knowing the basic syntax of the language, it is hard to know where to start.
Should I carry on learning JavaScript to finish the course and get to grips with things, or should I move more on to a C based language straight away? As from what I gather, to understand hlsl I really need to learn C/C++. But I don't know what book or tutorials to go for to learn enough without wasting my time knowing the ins and outs of the language if it has nothing to do with shaders, or in fact do I need to learn the whole of C++ to learn how to write good shaders?
I have been looking at getting Accelerated C++ to learn it. Also the Microsoft DirectX 9 programmable graphics pipeline, as this sounds like it would teach me more on the Mathematics behind it. Can anyone recommend these books or suggest others.
I am fortunate enough to currently be working as an artist at a studio, so I don't need to learn these things to get me a job, I just want more knowledge on the matter to hopefully move into a more technical artist position
Replies
The biggest hurdle for a lot of people is not having the technical background information you need to learn quickly. It's important to get an idea of 3d maths and how the rendering pipeline works if you want to do anything simple. If you don't know that well enough, you'll find it much harder to make sense of the shader code when you're just tinkering.
I'd actually recommend getting started with a few shaders in UDK. That doesn't teach you the language, but it does teach you some of the techniques you'll need to use. Then you'll be ready to just play around and try to replicate the same UDK shaders in CG/HLSL.
Is there anything big that's stumping you, or is it just kind of everything that you need to learn (as in, you don't really get any of it?)
I have a basic understanding of the maths behind it, like I know about matrices, vectors, dot products etc. Although I know about them I don't fully understand them all, nor could I write and solve an equation on paper with them without much help.
There isn't anything really big getting in the way, I am just strugglng to understand how to learn the syntax. Getting stuck into .fx files is part of what I am really interested in, so I just need to figure the best way to start learning. Just looking at files doesn't help me, because I don't understand how parenthesis etc. But using UDK is a good idea.
So would you suggest getting started with CG tutorials to get me used to the syntax?
Don't get scared/confused by these complicated sounding articles that try to explain the vertex-pixel shader pipeline in a lot of detail. They might make it sound like it's all about this complicated big system and how it ties into the graphics hardware. That stuff is mostly written by hardcore programmers and doesn't matter much for someone approaching it from an artist perspective. I know that's how they tried to teach me shaders at first and let's just say it never got me very excited...
The best advice I can give is to use a node-based system that helps you practice the core logic principles behind shaders. They hide the code at first and allow you to focus on what you exactly want to do without much burden. It's also much easier to analyze an example made by someone else, as you just follow the nodes and connections.
So to recommend you a start: get ShaderFX for max. It's a better choice than UDK, because it allows you to directly export the code and read it. UDK never allows you to move past the node stage, ShaderFX does. You can take a bare, solid color shader and export it, see what the code actually looks like for such a minimal base-frame. You can create some simple logic, like sampling and recoloring a texture, and export it to see what that looks like translated to HLSL.
In short, it's a translator for node logic to actual code; do it enough and you won't need it anymore eventually.
And to get back to that first point, the whole pixel-vertex-technique setup; ShaderFX generates that for you, and you could just keep working from this one "blank slate" code file for every new shader. It's kind of what I do.
but after that, if you still want to learn more about HLSL basics and the core syntax I highly recommend these DVDs by Ben Cloward -> https://www.cg-academy.net/es_catalog/product_info.php?products_id=64
They're awesome and will definitely get you pointed in the right direction!
edit, some more stuff:
The official Microsoft HLSL reference on MSDN. Quite technical and not always easy, but the best source to find clear answers on the syntax i think.
Basic, old article I wrote on color math. It's not really up to standard anymore, but it should help a bit for the very basics. I promise, I want to rewrite and continue this stuff this year!
And perhaps allow me to add my opinion on HLSL: I think it's the easiest, least nonsense and effort language there is. Once you feel at home in a shader's standard structure it's really very fun and satisfying to work with (just looking at that MSDN page makes me want to write stuff again haha). No setup required of program API's, no headaches with memory management, no compile steps, just CTRL-S, core functionality relatively portable between applications, etc...
Here are some additional links:
http://eat3d.com/shaders_intro
http://vimeo.com/cgbootcamp (has some CGFX for Maya tutorials)
Once I get that installed I shall start having a play around so I can visually see what is changing. I'm sure it wouldn't take too long to pick up the syntax and if I do have problems then I shall use the resources provided. Funnily enough I already had that colour math link bookmarked, I knew I would need it some day
I shall also look into getting those DVDs, they definitely look worth it.
If anyone else has any more great links or info please post, I am trying to soak up as much info as I can, but what has been posted already is a fantastic start. Thanks all for helping
Totally this. I often prototype shaders in the material editor simply because it's a very visual way of 'doing the maths'. You can learn a lot by fiddling with things in the material editor.
I did however find a link that still works, from ages ago. Take note that ShaderFX was provided with a free license for personal use, so this should be OK I hope...
http://www.lumonix.net/dlsfx385941.php
(no 2013 support, but you should really not be using that anyway, it's completely broken in terms of tangent display on shaders)
It might disappear, or maybe Lumonix is no longer OK with distributing it, in which case I'll remove it. Would be very unfortunate though, precisely for people like you
That is a shame that it has been discontinued, hopefully continued use will not be a problem.
Also, Kees visits these forums, maybe he'll pop in to tell us what's up.
Autodesk bought ShaderFX.
Currently working on the next real-time shader editor at Autodesk (If you are on the Maya beta you can try it out and give feedback).
You are welcome to use the old version via the download above.
Just keep in mind there will be no re-compiles of the old version.
But it should still help to get started seeing how the code works.
Eventually (I hope) Autodesk will provide something better to replace it.
** REGARDING BENS DVD: **
Please note that it is my understanding that the person hosting those DVDs never paid the people who made the tutorials (including Ben, Paul Neale, Bobo, etc). So please check with them first before you put more money towards a person who doesn't seem to pay our respected community members.
I am not fully up to speed on what happened there, but I recommend you contact Ben directly first.
Thats shitty to hear Kees. Thanks for the heads up. I really enjoyed the series when I was getting into shaders, shame to hear that
Jacky: Not sure how much this will help you but I asked this question like 3 years ago haha. Though mine was kinda about how to get in as a tech artist. Shaderfx is an awesome platform to start learning about shaders. Also once you understand the basics, you can start and port one of them by writing it yourself. I learned a lot from UDK too though its pretty limited once you start to implement other shader models. Someone once told me that writing shaders is like cooking. You have your texcoords, normals etc as ingredients and depending on what you do with the cooking process you can make some awesome stuff
Thanks for that link haiddasalami! Looks like it has a lot of info that I would be interested in as well. I shall have a hunt through
I also found this website Shadertoy. Has anybody had any experience with this? It looks like quite an interesting way to learn and test shaders if I am away from my main machines especially.
That Shadertoy website looks cool, but honestly, I wouldn't dabble with that yet, it's very different from how things work in Max with ShaderFX. It looks like it's GLSL and there's no real geometry; everything looks generated. Man, I'm having a hard time figuring out myself what's going in almost anything on that site...
I definitely learnt a lot from it!
http://eat3d.com/shaders_intro
But I would also go through UDK's material editor, it will teach you a lot about shader math and all the FX's you will want to achieve. then you can lookup how to do this in the code.
This is also a good resource, it's a bit difficult to set up. but it's quite a nice shader introduction tutorial. http://rbwhitaker.wikidot.com/hlsl-tutorials
You could try authoring your shaders in FXComposer https://developer.nvidia.com/fx-composer
But FX Composer is a bit 'full on' visually and it's not the most friendly program to use.
I personally author my shaders in Notepad++ with a syntax highlighting plugin and load my shaders into Maya with the HLSL shading node plugin (comes with Maya and just needs to be loaded in the plugin manager). The Eat 3D tutorial goes through setting up lights for Maya/the shader etc...
Anyway hope this helps, it may be a little bit of a mind dump/rant. But good luck and feel free to ask any questions here. I won't cringe and hopefully some of the others won't.
UDK is a good start; but it doesn't allow straight translation to code, which is really crucial to make the step away from nodes. It's also limited since it presents you a partially fixed shader, you can't easily go and replace the lighting model for example.
I wouldn't recommend FXComposer, never been a fan of it. Sure it may have some debugging options but it's too complicqted if you're coming at this as an artist.
I just do the same as DNC: Notepad++ (with custom HLSL syntax highlighting) and then just load it in Max. I actually went without Notepad++ for at least two years lol.
I know my biggest pita was actually the app-side of things & how to setup all the structs and so on. Once you have a simple framework it is more easy somehow to noodle with values etc. and get more comfortable with the meat and veg of shader programming.
And thanks for the advice on Shadertoy, looked really nice, but will stay away from it until I can write my own flaming shader
I have notepad++ installed and ready to go, I will try and learn how to easily iterate with it and max over the next few days. I was also thinking of starting a "progress thread" of my learnings. That way I can post all my horrible code for people to give me feedback on. Is this a good idea? If so, where would I post it, in Tech Talk?
https://docs.google.com/file/d/0B_bnXxEHXEanZW9BUUIyc1d5Ums/edit?usp=sharing
Another good way to learn stuff, especially application specific things, is to look at shaders other people have made. I've used Xoliul's shaders as reference when I couldn't get cubemapping or lighting to work in Maya. But do go and try to understand what is actually happening...
What Kees said about the DVDs is true. I haven't been paid by Cg Academy for several years now. I don't know if the company is still operating or not. I was never contacted by the owner to know what was going on. He just stopped replying to my emails and stopped sending my share of the proceeds.
Since then, I've received several emails from people that are interested in getting the DVDs because they'd like to learn. I'm really conflicted about what to do to be honest. I'd like to just give them the material to help them learn - or sell them directly, but I signed a contract with Cg Academy that prevents me from doing that - and I believe in upholding my side of the bargain. I'm kinda stuck - and it's frustrating.
Maybe if there's enough interest, I'll re-record the material and publish it on my own, or through another company.
And regarding "baseframe" shader, the great thing about ShaderFX is that you can generate it for multiple applications and languages from the same nodes, great to compare the differences. On that note, another very useful, free application is SourceGear Diffmerge. It compares two text files and highlights the differences in a very user-friendly way (Np++ does it too but not very well imo). You can also easily transfer code between the versions.
What it's useful for, is seeing what the exact code changes are when you change your nodes and re-export.
Oh and finally a tip for ShaderFX: when you export simple shaders, it will try to add lighting code to it (with things like attenuation for lights and so on). While this is a great system, in the beginning it's best to turn it off since you'll have some more advanced code in there that doesn't do anything. I'm sure there's an option to tell it to not inject that code.
That is a shame to hear Ben. I am a bit conflicted whether to buy them or not if the payment isn't funding the right people. So I would definitely be interested in a new DVD if you did, especially as yours is what seems to get brought up over and over even after all these years.
I shall do some studying and hopefully show off my work over the next few weeks, thanks again for the help and support
I've got a question about HLSL rendering in Maya and renderers in general.
I'm intending to make a skybox for my showreel an I wish to do it in HLSL instead of UDK. I looked into using Maya but I couldn't get transparency to work properly, it renders the background but does not calculate correct transparency between two objects with HLSL shaders on them.
I.e. I have a mesh with a solid red applied and no transparency. On top of this I have a blue mesh with 50% transparency. But when I look through the top I don't see a blend between the two meshes I just see the background colour.
Does anyone know of a solution to this in Maya?
If this is a known problem, does anyone know of any renderers that calculate multiple objects transparency correctly (I tried FXComposer but that didn't work as well).
Thanks
I have started following the Eat3D CGFX tutorial DVD and have finally got around to making my fist shader! I have created it in notepad++ C# sytax highlighting seemed to be the best. Unless anyone else has any better suggestions?
I pretty much understand it all, early days still though.
string ParamID = "0x002"; //it said this needed to be here for Max, why? float4x4 WorldViewProjection : WORLDVIEWPROJ <string UIWidget = "None"; >; string name = "A Super Simple Shader"; //he said this was just a maya thing, I put it in anyway to see if it affected Max in any way float3 shaderColor : DIFFUSE //is "shaderColor" different to "shaderColour"? < string UIName = "Shader Colour"; string UIWidget = "ColourSwatch"; > = {1.0f, 1.0f, 1.0f}; struct a2v { float4 position : POSITION; }; struct v2f { float4 position : POSITION; }; v2f vShader(a2v In) { v2f Out; Out.position = mul(WorldViewProjection, In.position); return Out; } float4 pShader(v2f In) : COLOR { float4 outColor = float4(shaderColor, 1); return outColor; } technique Opaque { pass one { VertexShader = compile vs_3_0 vShader(); PixelShader = compile ps_3_0 pShader(); } }Nothing is copy and pasted, but I have essentially just followed it word for word. He says this is CGFX and not HLSL, is there anything that anybody would do differently for HLSL? or just in general?
Also I wrote a little comment about "Colour" and "Color", for writing code should I just have to get into the habit of writing "Color" everywhere (shudder :P) so that it is universal? or is there even certain bits that HAVE to be written Color?
I know this might be a bit boring and simple for some of you no, but if you can stick with it for a while, your knowledge will be greatly appreciated down the line!
I'm a bit confused why you think the skybox is related? Because it's the thing that's drawing through? It's probably because your skybox was either the first or last thing created (depends on how the app sorts) that it draws it in front of the rest when transparent.
Jacky:
looks good. I'm not sure how useful a word-for-word copy is once you get into advanced logic instead of a baseframe like this (as in, you not thinking about WHY something is being written anymore because you're just trying to keep up with his code), it's probably more interesting to try some nodes or just dive into writing your own ideas straight away. Same with any tutorial I guess: just like I've always found it a bit weird and not so useful when people try to create the EXACT same 3d model as the guy in the tutorial is doing. But yeah for the first steps it's fine, so dont worry
Regarding C# syntax: just google for Notepad++ HLSL or CGFX syntax highlighting, it's pretty easy to set up.
And CGFX is pretty compatible with HLSL. This looks like you should only have to change the compiler ParamID to 0x003 to get DXSAS for Max, rename it to *.fx and it'll work.
And you seem to think that parameter naming matters for the code? The compiler doesn't care what you call a parameter (as long as you dont use illegal names or reference non existing ones), so write color, colour or whatever you want.
What does matter, is the semantic you type after those params, the " : DIFFUSE" part, those are pre-set, strict rules that need to be followed. Though on the other hand: a lot of apps don't use semantics at all (so you could just drop them) and they only really matter for your Vertex-2-Fragment struct (compiler is super-strict about proper semantics in there).
Aaah ok, I wasn't sure what parts can be called whatever you want (within reason) and what needs to follow strict rules. I understand a lot more now though. What is the 0x002 and 0x003 referring to? I saved that exact code, as an .fx file, and it worked ok.
They use a byte notation because you just want to use the simplest possible type of variable to determine this.
I think there's a list somewhere which compiler is which in Max, but 0x003 is the DXSAS (DirectX Script Annotation Standard) compiler, which is in short "the best one" for HLSL. I'm not too sure about CGFX, but I don't like using that because the error reporting is much worse, it barely gives you what line it's failing on. That's a tip for you too btw: CGFX and HLSL are interchangeable, but HLSL is easier to work with in Max.
it also seems to be a kind of plane as you can rotate it and move it about. Any ideas why this is happening?
I also completed the next part of the tutorial, just a lambert shader. One of the steps was to display the world normals, but they weren't showing the same as his. I completed the shader just to see if the end product worked, but I wanted to know why it wasn't displaying correctly. (With the code in the PS uncommented and worldNormal deleted it does show a lambert properly, I just commented quickly to test)
string ParamID = "0x002"; //Xoliul says this is a better one to use. float4x4 WorldViewProjection : WORLDVIEWPROJ < string UIWidget = "None"; >; float4x4 WorldInverseTranspose : WORLDINVERSETRANSPOSE < string UIWidget = "None"; >; float4x4 ViewInverse : VIEWINVERSE < string UIWidget = "None"; >; float4x4 World : WORLD < string UIWidget = "None"; >; ////////////////////////////////////////////////////////////// // Parameters section float3 shaderColor : DIFFUSE < string UIName = "Shader Colour"; string UIWidget = "ColorSwatch"; > = {1.0f, 1.0f, 1.0f}; //This is default value float3 lightDirection : DIRECTION < string UIName = "Light Direction"; string Object = "TargetLight"; int RefID = 0; > = {0.0f, 1.0f, 0.0f}; float3 ambientColor : DIFFUSE < string UIName = "Shader Colour"; string UIWidget = "ColorSwatch"; > = {0.15f, 0.15f, 0.15f}; //This is default value ////////////////////////////////////////////////////////////// // Structs section // input from application (Application 2 Vertex) struct a2v{ float4 position : POSITION; float4 normal : NORMAL; //float4 for multiplication purposes in 4x4 matrices }; // output to fragment program (Vertex 2 Fragment) struct v2f { float4 position : POSITION; float3 worldNormal : TEXCOORD0; //worldNormal becuase there can be different normals. TEXCOORD0 generic symantic }; ////////////////////////////////////////////////////////////// // Vertex Shader v2f vShader(a2v In) { v2f Out; Out.position = mul(WorldViewProjection, In.position); Out.worldNormal = mul(WorldInverseTranspose, In.normal).xyz; //Don't need to worry too much about what WorldIT is, just know when to use it. The .xyz part means it is just a float 3 (rgb is also valid. it just means ignore the "a" part of rgba) return Out; } ////////////////////////////////////////////////////////////// // Pixel Shader float4 pShader(v2f In) : COLOR { float4 outColor; float3 worldNormal = normalize(In.worldNormal); //normalise just keeps the vectors at max length of 1 float3 lightDir = normalize(lightDirection); //float3 lambert = saturate(dot(lightDir, worldNormal)); //saturate clamps between 0 - 1 outColor.rgb = worldNormal; //(lambert + ambientColor) * shaderColor; outColor.a= 1.0f; return outColor; } ////////////////////////////////////////////////////////////// // Techniques technique Simple { pass one { VertexShader = compile vs_3_0 vShader(); PixelShader = compile ps_3_0 pShader(); } }Where as his looks like this
The tutorial comes with 3dsMax versions of the files, so even though I am following him in maya, I also look at the Max source code files that are provided as some things are slightly different.
outColor.rgb = worldNormal;
to this:
outColor.rgb = worldNormal.xzy;
See how it's swapping the Y & Z?
edit: Although saying that, I just tested the code that I posted and it appears correctly on my work pc (although swapped round). Further investigation is needed. Thanks anyway
Edit again: It's because of Gamma, I didn't realise I must have it turned on at home, it is off at work, that's why it displays correctly.
If you see this sort of stuff, where the geometry is just going nuts, it means that the very basic code that converts the raw 3d-model coordinates to its screen-correct transformation (with perspective and everything applied) is not working like it should.
If you want to dive into actually finding the problem (not too easy, but a good excercise), you should get a simple shader example that works on DXSAS / 0x003. ShaderFX can do that for you. Then compare the application input and vertex shaders, try to see what matches and what differs, and find out what you need to do to get it to work.
Again: if you're gonna do HLSL shaders for max, you probably want to work with DXSAS, it really is the better choice.
And regarding Gamma; it really is one of your worst enemies when doing shaders in Max
ShaderFx is awesome, but it's crashing to often to experiment with it
Is there any other node based shader editor out there besides UDK ?
As for a 'unique' stand-alone node shader creator, nope, there isn't one, and I'm honestly surprised.
That is interesting to hear that there aren't more node based editors about. Especially as something like UDK works so well with it. It would be great for Marmoset to get a node based editor as it just seems to make everything more flexible. Maybe I could use this as my long term goal, make an Uber stand alone node editor.
Well a man can dream!
Oh and if anyone uses twitter, I would be grateful if people with hlsl knowledge gave me a follow. @JackyBoyII Just so I can ask little questions without having to create a forum post
Get it here! https://dl.dropbox.com/u/8510078/userDefineLang_DrewsHLSL.xml
Also I am quite proud, I managed to figure out why 0x003 wasn't working! it was in the Vertex part, v2f:
Originally it was
Out.position = mul(WorldViewProjection, In.position);
Out.worldNormal = mul(WorldInverseTranspose, In.normal);
But needed to be:
Out.position = mul(In.position, WorldViewProjection);
Out.worldNormal = mul(In.normal, WorldInverseTranspose);
Why would the multiply need to be in a certain order to work? (my simple understanding of multiplication being "4*6 = 6*4
multiplying a vector by a matrix, returns a vector.
I'm not sure what (matrix * vector) is but multiplying two matrices of different sizes gives you different results depending on the order.
This 'simply' has to do with the matrix order. The majority of engines use "Column Major" Packing/Ordering by default. If they don't it could be a programmers mistake or preference, or compiler... It's like this in HLSL and GLSL.
This means you would do mul(Vector, Matrix); or in GLSL (Vector * Matrix)
For Maya it would be Row-Major, so the opposite by default... You can choose to compile with DXSAS, which I believe likes Column-Major.
Hope this helps! ( and isn't confusing )
Some more info: http://en.wikipedia.org/wiki/Row-major_order#Column-major_order
edit: This is very important when you transform position of a vertex to homogeneous space. ( the semantic "POSITION" ).
Learning a bunch more of the basics atm so as I move on in tutorials I should be able to take it in a lot easier, then as Xoliul suggested, I will be able to follow a tutorial but not have to copy exactly.
just load the shader manually and everything works fine :-) (for max 2012 64bit)
I currently want to do a simple cubemapping shader with shaderFx, but somehow it looks not the same way I was expecting it to look like (as in Xoliul's shader): Mine looks curved?
Just some general information: that shaderFX download on page1 doesn't seems to be
the free version, its a trial. Now it wants a key or it closes itself.
Where I can get this free license key, I haven't found anything in the readme file ?
I figured out a solution to my alpha sorting problem, it was application based and in Maya I couldn't think how to change the draw order (or more specifically the sort priority). But I found that you can change the Depth Bias in the technique.
Using:
DephBias = 0.0f;
This needs Z testing and Z writing enabled to work, but it is the solution to the problem in Maya. I could find any Maya overrides.
@The Flying Monk
I also didn't realise the exact reason why the mul() function needed to be a certain way around, Thanks!
I found some more information on it over at the HLSL MSDN page http://msdn.microsoft.com/en-gb/library/windows/desktop/bb509628(v=vs.85).aspx
@NBLM
Did you manage to find a solution to your reflection problem, I would have agreed with equil's comment about the vector being in the wrong coordinate space,
'object->world' or at least something very similar.
and JackyBoy, how you getting on with Shaders???
I stopped just following tutorials and have been learning more about the syntax and what stuff is doing. For example I didn't know what the "In" (or Out) part of everything meant. As a beginner I didn't realise that it was something called In that you had named, I thought it was a special function to put something "In" something. A lot of tutorials don't seem to explain this stuff.
I will update soon with more interesting pictures!