Your a game artist, being concerned about performance is part of your job. Also if you are not the lead or art director you shouldn't be asking about this in an ideal situation anyway. You should be able to go to your Lead or Art director and tell them why you need this and also have a few examples that are relevant to your project to put it in context. I'm sure if you make a good enough case, the programmers will figure out a way to do it so it doesn't kill the performance.
Ok, so there seems to be some confusion and assumptions about how it's done currently at epic so I'll shed some more light on how shaders are authored.
The basic idea is
1. Make it look as awesome and as cool as shit as possible, this means anyone is making materials, doing what they need to do to make shit look good. All of our artists are experienced, smart, and everyone trusts everyone else to make stuff that looks good but isn't too unoptimized. I guess I'm kinda spoiled from working at Epic but it does kind of churn my stomach when I hear some of your guys talking about limited what people can work with even if it's shaders. I mean UE3's material system can be as easy or as complex as you want it to be, hell you can even right HLSL IN the material editor now! We even make it a habit to share new techniques for doing things in the material editor . Anyways yeah, #1 is make it look as awesome as possible, if you cant make it good with no constraints then you have a real problem, AND it's easier to optimize something than it is to make it better.
2. Make it run. It varies when this happens, sometimes it's when you're done with the set, when it's put in the level or when it ships, but it only happens if it's A. apparent its out of hand performance wise, or B. We've done a PIX capture on xbox and identified it as a culprit of performance hogging.
We have tons of custom shaders, I mean so many of the materials I made for the rift worm level in Gears2 were one off materials that just looked cool (I thought). I mean, when the blood rises up the characters in that level get blood on their pants that fades over time, That's all content driven! There was no programmer involved at any point, the material is assigned with Kismet and driven using Matinee. Yeah we cut out a lot of shit but there's a ton we don't cut. And so many LD's make their own material variants for an asset that'll have some snow, or a special damage look for their level or even a special rim that only happens in a certain direction on their map.
Back to the trust issue, seriously artists should be learning together and understanding how shit works and what runs slowly, and if they don't know they should ask. I think people should be given moderate to low restrictions when trying to make the art because it should look as good as possible.
To tackle some of the points against content driven shaders:
a) it's not practical performance wise as you know
-Not true, UE3 uses artist created shaders and we've always been able to optimize our games for shipping. Programmers can also go back and optimize code paths that the shaders generate. A LOT of work went into making sure UE3 materials are really optimized.
b) it can easily create a too high diversity of looks, in the negative sense
-I've worked on plenty of projects now with artist created shaders and this is the last thing you have to worry about especially if you're not doing anything too Non-Photo-Realistic
c) typically a good set of constraints, means focus, means "better production" imo. if you are allowed to do anything, you get lost
-Again not true, I'd rather see some kick ass material effect someone comes up with and we can spread throughout the game rather than them never playing with it at all. And from the games I've worked on with artist driven materials I've yet to see someone get lost making random shaders, I mean we trust each other enough and leave our ego's behind that we can ask for help if we hit a wall.
Your a game artist, being concerned about performance is part of your job. Also if you are not the lead or art director you shouldn't be asking about this in an ideal situation anyway. You should be able to go to your Lead or Art director and tell them why you need this and also have a few examples that are relevant to your project to put it in context. I'm sure if you make a good enough case, the programmers will figure out a way to do it so it doesn't kill the performance.
Very true, unfortunately where I work that doesn't work
I got somethings myself showing and proving, which was not that easy. At least I got something.
Epic is in a unique situation that very few places are, in that they can afford the salaries/time needed to ensure every hire is an incredible rockstar. Most companies you will find a couple amazing artists, a large chunk of competent, but not too technical artists, and then a few junior level people that are just there to get stuff done. Or you could look at games with extremely limited shader authoring. Not everyone is using UE3, a lot of engines only take purely HLSL written shaders, and I do think that's unreasonable to expect every artist to know. Especially since it does still require a level of technical creativity, that not everyone with HLSL writing skills comes up with worthwhile visual shaders.
This is why having shaders pre-authored, and documentation on exactly how to use it, is really important. It's great if you're at one of the handful of companies that uses an engine where shaders are really optimized, and every artist is top of their game, that's probably ideal, but there are hundreds of companies not in that position.
Interesting stuff Jordan, thanks for the clarification.
I totally agree that artist driven stuff is the way to make sure that everything looks as good as possible (although I have to say that if all the artists at Epic are comfortable making materials and shaders which both look good and are optimised then either you've hired a very select bunch of people, or there's been a fair bit of training time put into it?).
Also, this is a fairly important point:
A LOT of work went into making sure UE3 materials are really optimized.
Presumably that's partly because the Unreal tech is licensed out so extensively, you have a core team of coders who are focusing on the tools and techniques required by all games studios out there and trying to really optimise that pipeline and performance. While this is very cool, I think it's not really a "common" thing in the industry - there are only one or two developers who really license out their engine tech big-time while also working on games internally - it's not surprise that Epic are at the forefront of this since it's partly driven by the business model.
Not every studio can afford to spend the time creating, maintaining, improving and optimising a completely open workflow for shaders, and while it'd be nice, it often seems like it could be something that you could sink months of work into for something that, in the end, isn't actually "necessary" for a project to ship.
Definitely on paper it's a cool thing and I'm sure most artists would love that kind of control, but for a lot of studios and projects I imagine it's simply not on the cards.
Edit: Heh, Ben said pretty much the same thing as me in less words
c) typically a good set of constraints, means focus, means "better production" imo. if you are allowed to do anything, you get lost
-Again not true, I'd rather see some kick ass material effect someone comes up with and we can spread throughout the game rather than them never playing with it at all. And from the games I've worked on with artist driven materials I've yet to see someone get lost making random shaders, I mean we trust each other enough and leave our ego's behind that we can ask for help if we hit a wall.
I have never said artist should not play with shader effects. it was not a good idea to make that 3 point list, cause what I wrote afterwards seems to be completely ignored. I do see shader authoring by artists as communication tool, in which it's really good and surely can enhance quality. What my concerns are about is more the fact that the materials/shaders are directly used in the final product. And you said yourself that a lot of optimization work is still done afterwards and Epic surely is the leading technology provider thanks to having both top engineers and artists.
That said, I don't have any industry experience as in shipped games, so my assumptions are purely based on impressions and my own technical knowledge. It also has nothing to do with distrust, but the fact that moving more responsibility to people, also means you have to have people to support that/want to support that. And here for the average game production, I am not sure it could be achieved so well all the time. Because as others said it, it assumes that these people are trained to do good stuff again, which costs money, which in case of the many "grunts" in art production, I fail to see being invested at that scale.
but well it's my fault making that sort of "list" in the first place, cause of course one can always isolate it, and say what the fuck of course artists can achieve each point...
I don't understand how a project would use one main shader? for example, doesn't having transparency in the shader mean extra code? most objects don't need transparency so would you not have a non-alpha shader and one with?
I think this is actually very common, to the artist all you have is one "uber shader", you can use the inputs from that shader and easily compile it down to more optimized shaders depending on what sort of inputs the artists gives, you might think this would result in 50000 different shaders, but in reality its just a few. Likely less than 20 or so for the entire game. You just have a few combinations of normals + spec + apha, no normals, no alpha, etc etc.
I think for the vast majority of assets i've ever created, i think maybe... oh less than 5% would have ever really benifited from using an artist driven shader system, if you really have to do a lot of work for every asset to make it look good, chances are your base shaders are pretty terrible, and you're just trying to cover up for poor lighting, etc.
Of course if you're working on a horror game where walls bleed and you've got all sorts of different effects on every surface you're going to need to do a lot of custom shader work, but for any vaguely-realistic looking project i think you can get by with one quality shader that has nice options, ie: normal, diff, spec, gloss, glow, alpha, parameters for Fresnel, reflection amount/blur etc. Use that shader for nearly everything, and the small % of special case assets get special treatment. I would think this would streamline the process quite a lot, and give the artists more time to actually create art content instead of spending so much time on shaders.
Creating a custom shader for every asset, or really even half the assets in the game seems like an incredible waste of time, i would be really curious to see how many of these shaders end up being very similar in the end, because i've never really worked on a project like that.
Yep, what EQ said. I know for a fact they did this on Battlefield Bad Company since it was in one of their developer papers. They just had an "Uber Shader" which could do 99% of what you'd ever want and it seemed to work pretty well.
We do that on airborn too, not that it makes it any standard then, i just wanted to collaborate our experience with that topic. We have only a few master shaders, basicly not only one because the env uses lightmaps and the dynamic objects don't use lightmaps, besides that we only have a few more shaders that are very cheap, one for unlit, shaders for moving or static mattepainting and as unreal doesn't support alpha blend or alpha test as a shaderswitch we have the matte shaders with blend and test. basicly the only shaders that are unique per object are particle shaders as particly subUVs are nor controllable via an instance.
I decided for Mastershaders and instances for one simple reason, if i want to change stuff like the basic shading, i don't want to tackle every objects shader, i just want to open the mastershader and tweak it there, so we decided what shaders we would need and then created the needed mastershaders, based on, for example characters that won't ever need lightmaps or vertexblending stuff, soft alphas etc. but their own custom shading method, which evolves from time to time, when we see that this or that effect works better for us.
If its good in the master charactershader we try it out in the env shader too, if it works there it will stay in there, if not, no problem, everything is instanced, we don't have to touch up every asset.
Also Material Instances makes it easier for the artist to change the textures etc. i don't want the other artists to always crawl through shader networks, which can get quite complex, a "diffuse" slot, a "normalmap" slot etc. is very is easy to use for everyone if you don't need a certain part of a shader you can just switch it off per object, like SSS or Cubemapreflections, not every asset needs those options.
Obviously things are going to work differently at other studios. My post was simply written to clarify how we do things at Epic because there was a lot of assumptions made in previous posts. I understand that a company writing their own engine has to have a programmer write most of the shaders and things have to be planned out far in advance and that it's a ton of work creating a content driven material system.
I think there's is a lot of assumptions that content driven shaders are slow to work with. UE3 works pretty much like maya when it comes to material creation. If you want a basic phong material that has a diffuse texture, a normal map, and a specular color you can add all of that to a material in about 1 minute and it takes less than 10 clicks (even less if you're using a material instance). But a lot of you are overlooking the "little" things that are easy to do that make content driven materials shine. If I want my emissive to have a panning texture in it, or to flicker over time, I dont have to worry if a programmer has implemented a shader with that, I can make it myself and it doesn't take any knowledge of math or programming to do it.
I think you guys are looking at "custom materials" as though they have to be complex when it could be adding a simple flashing light or tinting the diffuse a color or adding a macro overlay texture, in a non-content driven pipeline you'd have to either plan this ahead and hope that it can fit into a programmers schedule or request it during production which means a programmer has to fit it between tasks and hope that it's high enough priority that they actually do it, when you could have made a material using the node editor in less than 5 minutes.
I can totally understand having huge master materials on Airborne, your game has a very specific look and most things are shaded specifically and if I remember correctly you use special lightmaps in the material. I can totally see using a master material for this. If I was developing this game at a studio though I'd say go ahead and make UE3 not default to phong lighting but to your NPR lighting and that would allow for some flexibility in materials and let people build on the master lighting model more.
And as a quick note Gears2 didn't have some crazy amount of people making materials and it was built in 2 years, the art team was pretty standard and when it comes to making custom materials a lot of people are capable, from LD's to environment artists. Yeah it can get hard when you're writing your own lighting model or trying to simulate SSS but most people aren't doing that, they're just trying to add that little flashing light for their level or the tv that has a moving picture and some noise.
However I think there's been a bit of a crossover here about what is a "material" and what is a "shader". It will probably be different depending on the tech of course.
When I talk about shaders, I'm generally not referring to simple stuff like making an existing texture input scroll/rotate/pulse/flicker, with our material system you can add that sort of thing to any of the "base shaders" simply by saying "do this to this texture input". Granted it's text driven at the moment (eg. "scroll ( time * 1 ), 0" to make something scroll in the U axis) but you don't have to be a maths genius to hook that stuff up, you just have to know the terminology (which isn't a problem if it's easily accessible and documented well).
A node-based system is sure to make that more accessible to the artist, but it's not out of reach on engines which have a "locked down" shader setup. What you're doing there is more material-based than shader-based.
In the Unreal Editor system it seems that the line is blurred between what is a "material" and what is a "shader" - eg. in many systems (like id tech) the "shader" is a base thing with all the really techy stuff written in the background by a programmer, and the "material" is the thing where the artist says "I want this texture and this texture and I want them to scroll and flicker".
That's probably why there's been a bit of confusion here, I think - we started out talking about shaders and ended up talking about materials - I guess in UE3 they're both regarded as one and the same? Or am I wrong there?
In our studio "materials" are regarded as entirely different to "shaders". I imagine the same is true for many other places too.
I totally agree with you about the flexibility and power of a nice artist-friendly node-based system though, just trying to clear up some of the confusion
Ah yeah, the lines are blurred with UE3. Programmers can write a shader (like HLSL) that generates the inputs for the nodes in our material editor, but we can also switch the material editor to use "Custom Lighting" Which means you can generate the whole lighting model in our material editor (some stuff is still done in code like indirect lighting since it uses spherical harmonics). You can use nodes to write you're lighting model or a custom code node to write HLSL.
assets that are likely to contain many different material types, color spec is generally needed. For things that tend to be a single material, obviously not so much.
You don't just let all the artists loose with tools and shaders, no limits, just because you "trust" them or something like that
sarcastic i hope? If someone doesn't understand limits then they're the issue, not the tools. Tools should never be limited, if someone makes something retarded it's cause they're retarded. I wouldn't want to limit what the brightest people could do just because someone would make something stupid.
Heh, Jordan's sandbox is so big he thinks it doesn't have edges... =P
Which is probably about just right.
You know I love ya buddy but you're still constrained in some ways, luckily for you those constraints don't get in the way with most things.
For some of us at smaller studios that lack WYSIWYG tools creating shaders can be a bit more time and tech consuming. Which can be overwhelming to artists and distract them from their number one job.
It's not necessarily taking options away from artists, but drawing up a list of what is needed and what isn't if that disscussion does not include artists I would consider finding another place to work.
the tools are limited by the money invested into them, which obviously greatly varies? so say you have some x people who are the brightest, you might get away giving them freedom with cheaper to make tools (good old text edit whatever). but the investment would only pay off in larger scale I think, the majority of people in asset production, probably wouldn't touch neither a fancy nor cheap tool. there is no sane reason for a "artificial" limit of the tools, but what they feature is a direct result of their investment, which would be wasted if it was locked? but maybe I miss the issue here.
sarcastic i hope? If someone doesn't understand limits then they're the issue, not the tools. Tools should never be limited, if someone makes something retarded it's cause they're retarded. I wouldn't want to limit what the brightest people could do just because someone would make something stupid.
Well having used Unreal to make Mass Effect I found the shader editor liberating, enabled us to create a range of cool shaders and great fx. Sadly we resorted to using standard shaders for a lot of the assets *loosing subtle elements that we had setup for specific assets).
Not gauging performance early on in development and ensuring that all departments are working within specific limitations is risky.
In the same way that a scene might have an assigned poly count max, it might also require clearly established boundary for the shaders. Everyone needs to work towards a specific performance target and obviously depending on the teams experience/art teams goals its clear there are multiple ways of handling this.
Having a generic shader that covers 90% of assets can be seen as a logical thing. Hell having a generic shader setup that deals with 50% is still worthwhile. (I made these numbers up)
Having everyone go batshit crazy and feeling broken hearted when the optimization stick comes around can be a real fucker and rarely at the point in a project do you have time to adjust the assets to the same standard, dealing with the impact the optimizations have brought.
If you have a very senior team or are making a sequel then the risk is reduced...
Definitely we benefited from the great tools, sadly good working practices weren't included with game engine! It takes a lot of planning and time to establish real performance metrics and likely these will be changing throughout development based on numerous factors including platforms and tech envolution.
Tech Envolution .... Core Tech - low level programming/ guts of your game
Here is a great example;
Your renderer has been rewritten for the 'X' time to work better for platform 'X' and catch up with the Advanced shadowing 'X' that game 'X' had. Game play system 'x' wasn't tested early enough and is having an big impact due to the extra number of 'X' it requires to work effectively. Now you need to optimize the shit of of assets 'X' to meet with the newly defined performance benchmark.
Its the classic circle of life as a game developer and although we shouldn't be looking over our shoulders trying to second guess everything, it helps to have some semblance of organization for when the shit eventually hits the fan and you need to ship the game!
I agree that having a core shader (or collection of shaders for characters, environments, etc) put together early on gives your art team a great base to work from. Special case needs like Jordan describes should be treated just like that: like special cases. If you have a toolchain like Unreal that allows artists to solve special cases on their own, all the better.
Art direction by shader is not the goal. If you have a shader that satisfies all of your needs for 90% of your assets, then you're not handcuffing your artists, just giving them a performant shader that does what they need.
Tech Envolution .... Core Tech - low level programming/ guts of your game
Here is a great example;
Your renderer has been rewritten for the 'X' time to work better for platform 'X' and catch up with the Advanced shadowing 'X' that game 'X' had. Game play system 'x' wasn't tested early enough and is having an big impact due to the extra number of 'X' it requires to work effectively. Now you need to optimize the shit of of assets 'X' to meet with the newly defined performance benchmark.
Its the classic circle of life as a game developer and although we shouldn't be looking over our shoulders trying to second guess everything, it helps to have some semblance of organization for when the shit eventually hits the fan and you need to ship the game!
Oh, i goggled "envolution" and i get legitimate results like "Envolution is a content management system that began as a fork of PostNuke." that seem like you're talking about a real thing here, lol
Oh, i goggled "envolution" and i get legitimate results like "Envolution is a content management system that began as a fork of PostNuke." that seem like you're talking about a real thing here, lol
Awesome, I'll inform the guys of the Next Generation Dynamic Content Management System we are implementing Internet posts are legally binding right?
Going back to many specific shaders vs a few generic ones and what Jordan and Spacemonkey say:
Working with UT3 assets I'm always suprised to see that there are more specific shaders in use than I'd expect. I think almost every content package contains one or more? For stuff like characters there's big master shaders, but a lot of props in UT use custom created shaders. This seems a bit contradictory with what spacemonkey is saying, that with Mass Effect they had to go back to standard shaders.
I just get the impression it's not such a huge issue in Unreal if you don't go completely overboard? For the Unreal powered game I worked on a lot of the environment artists created custom materials for almost every separate prop, completely unnecessary but still it ran fine. I think I even was the first one that started pushing the use of a set of generic shaders back then, after seeing a video of a presentation from Epic explaining the benefits of this system.
what I am saying is that a team needs to be organized enough to know its limitations. If your not using a deffered renderer, dont have massive issues with texture lookups or the number of shaders/functions per shader then why limit yourselves? Work to your limitations but also work in a flexible, accessible manner.
The tools in Unreal are empowering, particularly the shader editor. However just because you have access to creating custom shaders does not necessarily mean you need to do so. Its easy to forgot that Unreal has a great system to instance shaders too
Why not spend time on creating a core set of high quality shaders that are then instanced and used by many other materials? Its easier to amend, have programmers update them or replace shader functions with custom nodes and its lends itself to good organization.
You have a system in place, new artists can learn this system and its easy for team members to access and modify others work if they are required.
You can still get your best artists to prototype different shaders/params/whatever and its then easier to apply these new modified shaders to all the existing assets.
Of course I am representing my viewpoint here. As a lead I have to consider that people may leave, optimizations may have to happen, multiple platform parity may not be achievable, game play expectations are met with the assets (visibility, game clearly communicated etc)..
The more systems and organization that is in place, the easier it is to deal with the changes game development inevitably throws at you.
I completely agree with you on the benefits. No doubt a set of 10-15, maybe 20 master shaders can be enough for an entire game's specific look, in many cases I can even see thing working with just 5 or so master shaders. On top of that it's so much easier to make changes to the materials as you only have to edit a few files grouped into one package/location vs lots of shaders spread all over.
I was more thinking about the pure technical side of its performance impact. And overboard meaning the largest majority of objects/texturesets has a custom material created.
I would say it's at least better if the master materials are broken up per set or level. When optimizing at the end of the game you're not optimizing the asset you're optimizing the level so if I have this master material that's only used in this level that's slow It's much easier for me to kill some stuff from the master shader in that level/set than it is to kill it in one that's referenced in many levels, because those others may not need the optimization.
I would say it's at least better if the master materials are broken up per set or level. When optimizing at the end of the game you're not optimizing the asset you're optimizing the level so if I have this master material that's only used in this level that's slow It's much easier for me to kill some stuff from the master shader in that level/set than it is to kill it in one that's referenced in many levels, because those others may not need the optimization.
Fo sho, it should be noted that I dont deal environment art and that totally makes sense!
Replies
The basic idea is
1. Make it look as awesome and as cool as shit as possible, this means anyone is making materials, doing what they need to do to make shit look good. All of our artists are experienced, smart, and everyone trusts everyone else to make stuff that looks good but isn't too unoptimized. I guess I'm kinda spoiled from working at Epic but it does kind of churn my stomach when I hear some of your guys talking about limited what people can work with even if it's shaders. I mean UE3's material system can be as easy or as complex as you want it to be, hell you can even right HLSL IN the material editor now! We even make it a habit to share new techniques for doing things in the material editor . Anyways yeah, #1 is make it look as awesome as possible, if you cant make it good with no constraints then you have a real problem, AND it's easier to optimize something than it is to make it better.
2. Make it run. It varies when this happens, sometimes it's when you're done with the set, when it's put in the level or when it ships, but it only happens if it's A. apparent its out of hand performance wise, or B. We've done a PIX capture on xbox and identified it as a culprit of performance hogging.
We have tons of custom shaders, I mean so many of the materials I made for the rift worm level in Gears2 were one off materials that just looked cool (I thought). I mean, when the blood rises up the characters in that level get blood on their pants that fades over time, That's all content driven! There was no programmer involved at any point, the material is assigned with Kismet and driven using Matinee. Yeah we cut out a lot of shit but there's a ton we don't cut. And so many LD's make their own material variants for an asset that'll have some snow, or a special damage look for their level or even a special rim that only happens in a certain direction on their map.
Back to the trust issue, seriously artists should be learning together and understanding how shit works and what runs slowly, and if they don't know they should ask. I think people should be given moderate to low restrictions when trying to make the art because it should look as good as possible.
To tackle some of the points against content driven shaders:
a) it's not practical performance wise as you know
-Not true, UE3 uses artist created shaders and we've always been able to optimize our games for shipping. Programmers can also go back and optimize code paths that the shaders generate. A LOT of work went into making sure UE3 materials are really optimized.
b) it can easily create a too high diversity of looks, in the negative sense
-I've worked on plenty of projects now with artist created shaders and this is the last thing you have to worry about especially if you're not doing anything too Non-Photo-Realistic
c) typically a good set of constraints, means focus, means "better production" imo. if you are allowed to do anything, you get lost
-Again not true, I'd rather see some kick ass material effect someone comes up with and we can spread throughout the game rather than them never playing with it at all. And from the games I've worked on with artist driven materials I've yet to see someone get lost making random shaders, I mean we trust each other enough and leave our ego's behind that we can ask for help if we hit a wall.
Very true, unfortunately where I work that doesn't work
I got somethings myself showing and proving, which was not that easy. At least I got something.
This is why having shaders pre-authored, and documentation on exactly how to use it, is really important. It's great if you're at one of the handful of companies that uses an engine where shaders are really optimized, and every artist is top of their game, that's probably ideal, but there are hundreds of companies not in that position.
I totally agree that artist driven stuff is the way to make sure that everything looks as good as possible (although I have to say that if all the artists at Epic are comfortable making materials and shaders which both look good and are optimised then either you've hired a very select bunch of people, or there's been a fair bit of training time put into it?).
Also, this is a fairly important point: Presumably that's partly because the Unreal tech is licensed out so extensively, you have a core team of coders who are focusing on the tools and techniques required by all games studios out there and trying to really optimise that pipeline and performance. While this is very cool, I think it's not really a "common" thing in the industry - there are only one or two developers who really license out their engine tech big-time while also working on games internally - it's not surprise that Epic are at the forefront of this since it's partly driven by the business model.
Not every studio can afford to spend the time creating, maintaining, improving and optimising a completely open workflow for shaders, and while it'd be nice, it often seems like it could be something that you could sink months of work into for something that, in the end, isn't actually "necessary" for a project to ship.
Definitely on paper it's a cool thing and I'm sure most artists would love that kind of control, but for a lot of studios and projects I imagine it's simply not on the cards.
Edit: Heh, Ben said pretty much the same thing as me in less words
I have never said artist should not play with shader effects. it was not a good idea to make that 3 point list, cause what I wrote afterwards seems to be completely ignored. I do see shader authoring by artists as communication tool, in which it's really good and surely can enhance quality. What my concerns are about is more the fact that the materials/shaders are directly used in the final product. And you said yourself that a lot of optimization work is still done afterwards and Epic surely is the leading technology provider thanks to having both top engineers and artists.
That said, I don't have any industry experience as in shipped games, so my assumptions are purely based on impressions and my own technical knowledge. It also has nothing to do with distrust, but the fact that moving more responsibility to people, also means you have to have people to support that/want to support that. And here for the average game production, I am not sure it could be achieved so well all the time. Because as others said it, it assumes that these people are trained to do good stuff again, which costs money, which in case of the many "grunts" in art production, I fail to see being invested at that scale.
but well it's my fault making that sort of "list" in the first place, cause of course one can always isolate it, and say what the fuck of course artists can achieve each point...
I think this is actually very common, to the artist all you have is one "uber shader", you can use the inputs from that shader and easily compile it down to more optimized shaders depending on what sort of inputs the artists gives, you might think this would result in 50000 different shaders, but in reality its just a few. Likely less than 20 or so for the entire game. You just have a few combinations of normals + spec + apha, no normals, no alpha, etc etc.
I think for the vast majority of assets i've ever created, i think maybe... oh less than 5% would have ever really benifited from using an artist driven shader system, if you really have to do a lot of work for every asset to make it look good, chances are your base shaders are pretty terrible, and you're just trying to cover up for poor lighting, etc.
Of course if you're working on a horror game where walls bleed and you've got all sorts of different effects on every surface you're going to need to do a lot of custom shader work, but for any vaguely-realistic looking project i think you can get by with one quality shader that has nice options, ie: normal, diff, spec, gloss, glow, alpha, parameters for Fresnel, reflection amount/blur etc. Use that shader for nearly everything, and the small % of special case assets get special treatment. I would think this would streamline the process quite a lot, and give the artists more time to actually create art content instead of spending so much time on shaders.
Creating a custom shader for every asset, or really even half the assets in the game seems like an incredible waste of time, i would be really curious to see how many of these shaders end up being very similar in the end, because i've never really worked on a project like that.
I decided for Mastershaders and instances for one simple reason, if i want to change stuff like the basic shading, i don't want to tackle every objects shader, i just want to open the mastershader and tweak it there, so we decided what shaders we would need and then created the needed mastershaders, based on, for example characters that won't ever need lightmaps or vertexblending stuff, soft alphas etc. but their own custom shading method, which evolves from time to time, when we see that this or that effect works better for us.
If its good in the master charactershader we try it out in the env shader too, if it works there it will stay in there, if not, no problem, everything is instanced, we don't have to touch up every asset.
Also Material Instances makes it easier for the artist to change the textures etc. i don't want the other artists to always crawl through shader networks, which can get quite complex, a "diffuse" slot, a "normalmap" slot etc. is very is easy to use for everyone if you don't need a certain part of a shader you can just switch it off per object, like SSS or Cubemapreflections, not every asset needs those options.
I think there's is a lot of assumptions that content driven shaders are slow to work with. UE3 works pretty much like maya when it comes to material creation. If you want a basic phong material that has a diffuse texture, a normal map, and a specular color you can add all of that to a material in about 1 minute and it takes less than 10 clicks (even less if you're using a material instance). But a lot of you are overlooking the "little" things that are easy to do that make content driven materials shine. If I want my emissive to have a panning texture in it, or to flicker over time, I dont have to worry if a programmer has implemented a shader with that, I can make it myself and it doesn't take any knowledge of math or programming to do it.
I think you guys are looking at "custom materials" as though they have to be complex when it could be adding a simple flashing light or tinting the diffuse a color or adding a macro overlay texture, in a non-content driven pipeline you'd have to either plan this ahead and hope that it can fit into a programmers schedule or request it during production which means a programmer has to fit it between tasks and hope that it's high enough priority that they actually do it, when you could have made a material using the node editor in less than 5 minutes.
I can totally understand having huge master materials on Airborne, your game has a very specific look and most things are shaded specifically and if I remember correctly you use special lightmaps in the material. I can totally see using a master material for this. If I was developing this game at a studio though I'd say go ahead and make UE3 not default to phong lighting but to your NPR lighting and that would allow for some flexibility in materials and let people build on the master lighting model more.
And as a quick note Gears2 didn't have some crazy amount of people making materials and it was built in 2 years, the art team was pretty standard and when it comes to making custom materials a lot of people are capable, from LD's to environment artists. Yeah it can get hard when you're writing your own lighting model or trying to simulate SSS but most people aren't doing that, they're just trying to add that little flashing light for their level or the tv that has a moving picture and some noise.
However I think there's been a bit of a crossover here about what is a "material" and what is a "shader". It will probably be different depending on the tech of course.
When I talk about shaders, I'm generally not referring to simple stuff like making an existing texture input scroll/rotate/pulse/flicker, with our material system you can add that sort of thing to any of the "base shaders" simply by saying "do this to this texture input". Granted it's text driven at the moment (eg. "scroll ( time * 1 ), 0" to make something scroll in the U axis) but you don't have to be a maths genius to hook that stuff up, you just have to know the terminology (which isn't a problem if it's easily accessible and documented well).
A node-based system is sure to make that more accessible to the artist, but it's not out of reach on engines which have a "locked down" shader setup. What you're doing there is more material-based than shader-based.
In the Unreal Editor system it seems that the line is blurred between what is a "material" and what is a "shader" - eg. in many systems (like id tech) the "shader" is a base thing with all the really techy stuff written in the background by a programmer, and the "material" is the thing where the artist says "I want this texture and this texture and I want them to scroll and flicker".
That's probably why there's been a bit of confusion here, I think - we started out talking about shaders and ended up talking about materials - I guess in UE3 they're both regarded as one and the same? Or am I wrong there?
In our studio "materials" are regarded as entirely different to "shaders". I imagine the same is true for many other places too.
I totally agree with you about the flexibility and power of a nice artist-friendly node-based system though, just trying to clear up some of the confusion
Truth!
sarcastic i hope?
Which is probably about just right.
You know I love ya buddy but you're still constrained in some ways, luckily for you those constraints don't get in the way with most things.
For some of us at smaller studios that lack WYSIWYG tools creating shaders can be a bit more time and tech consuming. Which can be overwhelming to artists and distract them from their number one job.
It's not necessarily taking options away from artists, but drawing up a list of what is needed and what isn't if that disscussion does not include artists I would consider finding another place to work.
Well having used Unreal to make Mass Effect I found the shader editor liberating, enabled us to create a range of cool shaders and great fx. Sadly we resorted to using standard shaders for a lot of the assets *loosing subtle elements that we had setup for specific assets).
Not gauging performance early on in development and ensuring that all departments are working within specific limitations is risky.
In the same way that a scene might have an assigned poly count max, it might also require clearly established boundary for the shaders. Everyone needs to work towards a specific performance target and obviously depending on the teams experience/art teams goals its clear there are multiple ways of handling this.
Having a generic shader that covers 90% of assets can be seen as a logical thing. Hell having a generic shader setup that deals with 50% is still worthwhile. (I made these numbers up)
Having everyone go batshit crazy and feeling broken hearted when the optimization stick comes around can be a real fucker and rarely at the point in a project do you have time to adjust the assets to the same standard, dealing with the impact the optimizations have brought.
If you have a very senior team or are making a sequel then the risk is reduced...
Definitely we benefited from the great tools, sadly good working practices weren't included with game engine! It takes a lot of planning and time to establish real performance metrics and likely these will be changing throughout development based on numerous factors including platforms and tech envolution.
Here is a great example;
Your renderer has been rewritten for the 'X' time to work better for platform 'X' and catch up with the Advanced shadowing 'X' that game 'X' had. Game play system 'x' wasn't tested early enough and is having an big impact due to the extra number of 'X' it requires to work effectively. Now you need to optimize the shit of of assets 'X' to meet with the newly defined performance benchmark.
Its the classic circle of life as a game developer and although we shouldn't be looking over our shoulders trying to second guess everything, it helps to have some semblance of organization for when the shit eventually hits the fan and you need to ship the game!
I agree that having a core shader (or collection of shaders for characters, environments, etc) put together early on gives your art team a great base to work from. Special case needs like Jordan describes should be treated just like that: like special cases. If you have a toolchain like Unreal that allows artists to solve special cases on their own, all the better.
Art direction by shader is not the goal. If you have a shader that satisfies all of your needs for 90% of your assets, then you're not handcuffing your artists, just giving them a performant shader that does what they need.
I liked it better when i thought it was a typo =(
Awesome, I'll inform the guys of the Next Generation Dynamic Content Management System we are implementing
Then my good man, you have not had the pleasure of using Tech Envolution yet!
Working with UT3 assets I'm always suprised to see that there are more specific shaders in use than I'd expect. I think almost every content package contains one or more? For stuff like characters there's big master shaders, but a lot of props in UT use custom created shaders. This seems a bit contradictory with what spacemonkey is saying, that with Mass Effect they had to go back to standard shaders.
I just get the impression it's not such a huge issue in Unreal if you don't go completely overboard? For the Unreal powered game I worked on a lot of the environment artists created custom materials for almost every separate prop, completely unnecessary but still it ran fine. I think I even was the first one that started pushing the use of a set of generic shaders back then, after seeing a video of a presentation from Epic explaining the benefits of this system.
what I am saying is that a team needs to be organized enough to know its limitations. If your not using a deffered renderer, dont have massive issues with texture lookups or the number of shaders/functions per shader then why limit yourselves? Work to your limitations but also work in a flexible, accessible manner.
The tools in Unreal are empowering, particularly the shader editor. However just because you have access to creating custom shaders does not necessarily mean you need to do so. Its easy to forgot that Unreal has a great system to instance shaders too
Why not spend time on creating a core set of high quality shaders that are then instanced and used by many other materials? Its easier to amend, have programmers update them or replace shader functions with custom nodes and its lends itself to good organization.
You have a system in place, new artists can learn this system and its easy for team members to access and modify others work if they are required.
You can still get your best artists to prototype different shaders/params/whatever and its then easier to apply these new modified shaders to all the existing assets.
Of course I am representing my viewpoint here. As a lead I have to consider that people may leave, optimizations may have to happen, multiple platform parity may not be achievable, game play expectations are met with the assets (visibility, game clearly communicated etc)..
The more systems and organization that is in place, the easier it is to deal with the changes game development inevitably throws at you.
I was more thinking about the pure technical side of its performance impact. And overboard meaning the largest majority of objects/texturesets has a custom material created.
Fo sho, it should be noted that I dont deal environment art and that totally makes sense!