Open World Level Design (think GTA)
We're doing grit game engine for a real streaming engine just like the ones used in Rockstar's games.
our programmer is concerned about batches and draw calls. He is very accomplished in the development world including professionally, though not for games, but he is very well versed, and he has been one of the people improving bullet and ogre for some time, so his information is surely not broken, but maybe there is something being missed.
I know for a fact every asset in Rockstar games uses many textures-per-model.
There is no way to atlas a building and have it look good AND be one or two textures max without the textures being huge.
So I'm at a standstill in my personal workflow, wondering how there can be so many different models, and all of them having so many different materials in gta?
our programmer is concerned about batches and draw calls. He is very accomplished in the development world including professionally, though not for games, but he is very well versed, and he has been one of the people improving bullet and ogre for some time, so his information is surely not broken, but maybe there is something being missed.
I know for a fact every asset in Rockstar games uses many textures-per-model.
There is no way to atlas a building and have it look good AND be one or two textures max without the textures being huge.
So I'm at a standstill in my personal workflow, wondering how there can be so many different models, and all of them having so many different materials in gta?
Replies
Texture sizes is not really an issue today, it's all about the total of memory you'll use, so bunch that shit together if it's all going to be in the memory at the same time anyway, downside is that you can't tile it, (or you can, if you decide to atlas a whole set of textures in only one axis, so you can tile in the other).
But if you atlas it in every axis, just split it up with edges anyway to tile it.
Reuseable textures, vertex colour for variation, as you said, there's no reason for everyone to keep making that same concrete texture, but you'll benefit from having all re-usable textures as one giant texture.
every car in GTA IV for example has like 17 maps it shares between all of them, and 4 to 6 maps per vehicles. They are all 256 or so, but still, thats a lot of maps for a lot of cars on screen at once.
Not to mention, the buildings are all like, if the texture is unique (even if tiled) it's in the model, and if it is shared around the world, it is in other wtd/archives.
So many textures - it really doesn't matter though?
draw calls, batches, etc - they say we cant do it because of that, but I see GTA did it with my own eyes - I am not a technical person with the back end of game engines or optimizing things, but I don't see how to get around the vast amount of textures that one could never bake. Even if we made atlases and tiled by geometry, it's still gonna be too too big of a texture unless you sacrifice quality, which is not an option either as it "needs to look as realistic as possible".
Racking my brain for weeks.
I mean, the same amount of textures, even if you atlas them together, will take the same amount of space, only it is one texture instead of several.
What hardware are you targeting?
Most hardware today should be able at a minimum to handle 4096x4096 textures, which mean, if you have every piece of reusable texture be 512, you could fit 64 textures in one 4096x4096 atlas.
Usually when they say "we cant do it" it means, "We can, but it comes with high costs"
In the end it's all about finding a way to get down the drawcalls in the scene as much as you can to the degree that is needed.
But still, don't forget about vertex-painting for adding shading and ambient lighting, it helps alot in these cases where you wont afford baking in that stuff, and it's practicly free.
This is where you are wrong. I guarantee you it DOES matter. They do some amazing things with their budgets, but they have budgets they follow just like everyone else. As someone who has built several cities for retail games, and studied the GTA games very intently in the process, I speak with some experience. There are lots of small tricks to save some draw calls, but there is no magic bullet to make you not have to worry about them anymore.
The first step is to understand them as best as you can. Make sure you have read everything you can find about what they are, and how they work. Here's a good place to start; http://wiki.polycount.com/Game%20Rendering%20Primers
Remember that there are quite a few knobs to turn and things you can balance. Good tools for seeing EXACTLY how you fare on draw calls, memory and instancing are VERY important.
Instancing is a very important thing to understand. If you have 20 identical buildings, do they count as one draw call, or as twenty? Or do they count as 3, or 11, and why? There are lots of engine-dependent things that can effect this, such as lighting, vertex-painting, LODs (number and type on screen), and many other things. You need to understand how these affect your draw calls, because it can be a big factor for you. Maybe you are making lots of LODs, and less LODs would mean more available draw calls. Maybe your LODs can trade memory for draw calls, by baking multiple textures down into one. Maybe you are using some type of lighting that costs you draw calls, and sacrificing some lighting (or even number of lights) can get you more draw calls. You need to know these things.
Good streaming for textures is also very important. I used to stress about texture memory a lot, but after a few iterations of our texture streaming tech I was able to trade memory for draw calls in some good ways. With the ability to handle more texture memory, you can use a pretty big texture for your buildings, and with a big texture you can usually fit an entire building into one texture. This is a big deal as buildings draw the most at your middle and far distances, and affect draw calls in a big way.
The other big point is to focus on the objects that you use the most. If you are using an object a whole lot (like a lamp-post, for instance), than make sure that object is using the least amount of draw calls possible (preferably one).
Shader-tricks are also very useful. You can use shaders, masks, and UVs to do all kinds of cool stuff. Cars and buildings can be different colors, but still share one draw call. You can do things like offset UVs randomly (to make windows that are on or off randomly, every time the building is drawn). You can have one building that shows up different colors every time it's placed, but they all count as one draw call. I have done these things, and you would be well suited to get your graphics programmers to explore these tricks as well. These are tricks that even valve uses in L4D, for both memory and performance options. Use shaders to get visual variety without sacrificing draw calls.
Hope this helps.
Also, we should get in PM boxes together or in chat and discuss GTA some. See if we noticed anything the other didn't etc, could be good for both of us, could help us a lot too.
Cheers guys, keep the info coming! We want this engine to be THE open source streaming game engine. Tall order, but our will is good
So I fired off an email to him last night to see what he can write up.
think about it....if each building is made of 20 or so shaders but is reduced to 1 at LOD and the LOD distance is tight....its all good in reducing shader counts
cman2k-"The other big point is to focus on the objects that you use the most. If you are using an object a whole lot (like a lamp-post, for instance), than make sure that object is using the least amount of draw calls possible (preferably one)."
depends actually a lampost would possibly need several shaders to acheive a good look without massive memory use and adding 1 or two draw calls for something used everywhere is not a big deal...more important is cutting down on the amount of shaders on more unique objects
also atlusing and streaming dont really go that well...shifting large packets in and out of memory is a bad idea
Optimizing draw calls of a unique object is going to save you some draw calls in one part of your environment only...around the unique object. If your lamp post is 4 draw calls, and you can cut it down to one, then you will save 3 draw calls EVERYWHERE, not just in one part of the environment.
When you are trying to optimize for draw calls I think your most-used objects are always the first place you will go, because they have the largest impact on your draw calls around the entire environment as a whole. If anything, one-off unique props provide the least bang-for-your-buck when it comes to optimization, because there is only a small area of the environment that will benefit from it.
If you can cut 50 draw calls from your most used objects, you will generally have around 50 more draw calls around your entire environment. If you cut 50 draw calls from unique objects....it will have very little impact on your environment as a whole, and you will likely only have a handful of extra draw calls on average in your environment in any given area.
Also I am totally confused by your statement that a "lampost would possibly need several shaders to achieve a good look without massive memory use"...how does "massive memory use" factor in here? Is packing everything into one texture sheet massive memory use? and how does using several shaders drastically change the amount of memory use? Are you implying lots of tiled small textures or something?
I can understand the statement that a lampost would benefit from more than one shader, primarily for glass & glowing stuff...but when you are building a city, everywhere you can squeeze some draw calls is worth it. Opaque glass that is masked to glow would be perfectly acceptable trade off if it meant an extra couple of draw calls across my entire environment.
and by massive memory saving i mean if you need at least 2 extra channels for those masks emmisive and alpha that is an extra map(rgb) or two(single channel) which would probably be an extra 25% memory cost and still be an extra draw call
anyway i think it depends on your engines use (or not) of batching draw calls
This is exactly what I assumed! When I study the GTA map, those guys did such an amazing job, and this makes a lot of sense. Because the LOD looks basically like just a directional shade on polys and diffuse.
I feel like this is a whole different ball game than other games. I figured big pieces would be bad, be they in texture chunks or huge models.
Huge, batched up models are bad too, right? These will cause the pop to from lod to close and back to be visible, slow, and ugly, right?
edit: must now read all that comes after this post.