Material editors vs. Custom shader code
polycounter lvl 11
So I'm mostly just curious as to how other studios handle this.
The tradeoff is between a node-based material editor, or having all the shaders created by a programmer. The advantage of an editor is the artistic flexibility that comes with it, the disadvantage is that it shoots out inefficient shaders.
How is this handled at your studio? Just one or the other? Or both maybe, where you use a material editor to make the material, then have a programmer optimize the shader once it's done?
The tradeoff is between a node-based material editor, or having all the shaders created by a programmer. The advantage of an editor is the artistic flexibility that comes with it, the disadvantage is that it shoots out inefficient shaders.
How is this handled at your studio? Just one or the other? Or both maybe, where you use a material editor to make the material, then have a programmer optimize the shader once it's done?
Replies
The entire 'nodes are inefficient' issue only exist because many who wrote said infrastructure didn't know what they were doing, and because they tend to leave behind alot of 'rubbish code' without a proper intermediate conveyer, to make the switch to shader types available (CGFX, HLSL, DXAS, etc), but thankfully (and hopefully) this will be thrown out of the window soon enough once ALL the engines actually optimize the stuff well under the hood.
As for the programmer and how they can save the day for nodes...well, I'm sure someone who actually worked in said area can answer you better.
However, the compiler is intelligent enough to turn it into very efficient shader code so other than a longish compile time (which appears to be cached as a binary? does some processing when moving to a machine with a different GPU) it doesn't even matter.
The bulk of the shader code which actually does the lighting and so on is kept out of sight which is a mixed blessing but I would not be surprised if this is hand written in GPU assembler or kept as a compiled binary and just linked to every material's code during the compile process.
I'd say you probably want most of the surfaces in your world to not have anything more complex than basic lighting and texture samples simply to save on fillrate cost, so again it doesn't matter too hugely what the few custom bits of shader code do.
should compile to something like
Also afaik all processing on the gpu is done on data sets of 4, so these 2 lines should be just as fast:
The flipside is that multiple passes, and other non-standard surface shading, is generally poorly supported (especially with Unity's Strumpy).
Ideally, if a couple of your guys in the team know how to code shaders, then it should be alright to let the artists roam free and happy with the node editor, then get the codemonkeys to do a final pass.
So basically, instead of having nodes, you would have editor under which you could put up words and what they all mean and have the engine load it externally as such, such as;
{Lambert + Diffuse Fresnel
Lambert = Normal (fDot) Light
{Diffuse Fresnel = Input some fancy code here.
So you would have the writer do all the work behind the scene, and the person, with a little knowledge of basic HLSL and CG, could get the same benefits of everything else.
Sadly, I can't find the topic, nor remember where I read about this, but the idea was very cool indeed, especially if you get tired of clicking all day long on nodes, or aren't too well versed in shader lingo to declare each time a parameter in the right place (especially when you have to expand the Normals).
Education meaning we talk to anyone who's going to touch materials and give an overview of what's going to slow down the system (so in Unreal material terms, what instruction count and material dependency mean, and how they're calculated). Then we let them go crazy.
Profiling comes in afterwards, when you're trying to get the game running at frame rate. Since 10% of the game is going to cause 90% of the slowdown, we run the game on PC and consoles and look at profiling tools to work out what's running slowest and causing bottlenecks. Fix that stuff and then ship it.
As for 'pure' universal translator which is easy to use and can be plugged in an engine? Don't think so.
Edit:
Yeah, there's a message on their site saying the guy joined Autodesk, so there's no more ShaderFX.
and it looks like mental images has stopped distributing mental mill too!
https://www.mentalimages.com/products/mental-mill.html
Render Monkey has sadly gone out of development. You can still find it though FXComposer is probably your best bet when it comes to creating HLSL/CGFX shaders or good ol notepad++.
From what I gather when I looked into shaders (this was when I was first researching tech art) some places have a material editor where a tech artist can prototype something and pass it off to the graphics engineer who will profile/covert etc. Naughty Dog showed theirs off or I think it was Sony who made it and first party studios have access to it. Looked good from the slides.
not quite what you want but have you looked at the source window in the material editor?
window menu - turn on checkbox marked "source"
* there is no such thing as shaders written in GPU assembler. There is something like am immediate representation which is what DirectX's HLSL compiler spits out, that is passed to the driver and then turned to true GPU assembler (as that is very vendor/hw specific there is no "standard").
* vector operations != scalar operations. The float4*float4 as fast as float*float is not globally true. NVIDIA has a scalar architecture for quite some time and AMD recently moved to one. Typically the compiler is good enough in dead code elimination. So if you do float4 op once but later say use only .x, then only that scalar will be calculated.
How good compilers are depends very much on the platform, your mileage will vary greatly on mobile platforms, while PC (especially DX) or consoles should be very tuned.
Personally I use my own scripted IDE for shader work, which passes files to offline compilers (GLSL,HLSL)
http://www.luxinia.de/index.php/Estrela/Shader
Yeah, I wish this was possible too. Then I could have people just make their materials in UDK, and then port them over.
ShaderFX was not abandoned.
It has been acquired by Autodesk and its development has moved there.
If you want to be on a future beta, drop me an email: kees.rijnen at autodesk.
As for the question about node editor versus hand-code.
It is mostly a personal preference.
From a code inefficiency point of view. if you would know what you are doing with hand-written code to optimize performance, you could do this too in a good node-editor (for the most part).
The compiler is also very good at optimizing.
Some people will never want to use a node editor, and some will never want to write code by hand.
As for code readability, I have spend a lot of time working on this in the new ShaderFX and it is doable to output code that is similar to what you would write by hand. But it does take a good amount of programmer time to get that to happen.
Unreal does have many #ifdef in the code and it makes it very hard to do anything with the code afterwards, but I don't think it was their goal for you to print out the code and hand edit it or to take their code elsewhere. So they did what worked for them. And it is an awesome editor.
For Mental Mill I always thought the code was very bad to do anything with and I thought that was a major fail for that editor because if you want wide adoption of your 3rd party, shader node editor people should be able to read/edit the code.
A node editor may never make code look exactly like hand written code, but it can do a much better job then Mental Mill did, imo.
About the Autodesk thing, you're not talking about Slate are you? At least I hope not. Or do you just mean that ShaderFX will be part of some next version of Max?
I can't say when/what/if anything gets released in any future version of 3dsMax. Sorry.
Best I can offer is to put the name of anybody interested on a "future beta list".
As for node editor code performance. I think you can get closer then 90% performance wise versus hand written code.
It all depends on your level of understanding of real-time shaders.
If you know what to do and what not to do with hand written code, you can apply the same knowledge with a node editor.
If you don't and start mixing 16 4k textures "just because you can", then yes, your code will not very fast
Thanks btw Kees.