Common shadow setups for mobile games?
vertex
Hello,
I was really happy recently when I got dynamic shadow mapping working in my mobile engine. It works perfectly for my 2.5d horror game as the viewable area is smallish, so the resolution of the shadow map is acceptable.
However - I'm also prototyping a mobile racer, and unfortunately my hopes for fully dynamic shadows were quashed in an instant
Even with a 2048 shadow map covering approximately half of the viewable area, the shadows are far too low res.
So I was wondering what a common shadow setup is for mobile games or even console games? Presumably lightmapping for most of the level, and dynamic shadow mapping in as small an area as possible near the camera, for the player and other nearby dynamic objects?
Incidentally I checked out GTA5 recently for the first time in a while, and I was impressed with how good the shadows are. There appear to be dynamic shadows on almost everything, and they are good quality too, better than most open world games. Amazing given the sheer scope of the environment and the age of the hardware it's running on.
Cheers.
I was really happy recently when I got dynamic shadow mapping working in my mobile engine. It works perfectly for my 2.5d horror game as the viewable area is smallish, so the resolution of the shadow map is acceptable.
However - I'm also prototyping a mobile racer, and unfortunately my hopes for fully dynamic shadows were quashed in an instant
Even with a 2048 shadow map covering approximately half of the viewable area, the shadows are far too low res.
So I was wondering what a common shadow setup is for mobile games or even console games? Presumably lightmapping for most of the level, and dynamic shadow mapping in as small an area as possible near the camera, for the player and other nearby dynamic objects?
Incidentally I checked out GTA5 recently for the first time in a while, and I was impressed with how good the shadows are. There appear to be dynamic shadows on almost everything, and they are good quality too, better than most open world games. Amazing given the sheer scope of the environment and the age of the hardware it's running on.
Cheers.
Replies
@RyanB has been posting some great optimization tips in the Unity section. For example:
http://polycount.com/discussion/182384/lighting-for-mobile-in-unity-optimization
I think you're right about the shadows. I checked out some mobile games, and even Asphalt for example appears to have just lightmaps with a fake shadow raycast into the floor.
I'm somewhat surprised though. On my mobile game I have fairly complex scenes with around 100,000 polys being rendered, with dynamic shadows being cast and received by everything, and a 1024 shadowmap running at 30fps on a Note 2. Crucially though, with hard shadows. Doing the shadowmap sampling and calculations once per pixel is doable, 16 times not so much.
I also played Real Racing 3 recently, and I was surprised to see real time environment mapping, running well even on a 4 year old phone.
That's something else I also got running recently, and unless there's a cheaper way I don't know about, it's expensive. You have to render the scene 6 times, once for each side of a cube map. Even if you use 128 textures, the extra draw calls and polys rendered is not exactly trivial. The only thing I can think of is they build some very low poly batched geometry specifically for reflections, and render that to the cubemap instead of the standard models.
You could also look into dual-paraboloid reflection mapping. Then you're only rendering two views. Up/down or back/front. There's a noticeable seam sometimes, but usually not that big a deal on something distorted like a car body. They might even be rendering only one view (facing back at the players camera).
Incidentally I had a look at Ridge Racer 6 the other day for the first time in years, and they had real time reflections yet no dynamic shadows. Again I was pretty surprised, as my preconceived notion was that it would be far more expensive than dynamic shadows.
Cheers.
There's a couple of possibilities I can think of.
1 - Just raycast towards the main shadowcasting light, and if you hit something, shade the character. Obviously a bit crap and limited, but should work.
2 - A better option would be to raycast down, fetch the triangle the ray hits, calculate the UVs of the intersection, then do a manual lightmap lookup to get the exact value of the lightmap that the player is on.
You could then feed this value to the shader, and add just one line of code to the fragment shader, something like
mul targetReg targetReg dynamicShadeReg
Oh and btw thanks to @Swordslayer I got a pretty good, fast shadow setup working for my mobile racer. Exported vertex colour lighting instead of lightmapping, it's just so fast and easy. Then a dynamic shadowmapper just for the car, so world geometry shadows get cast onto the car.
Unfortunately though I'm having trouble getting the in game shadowcasting to match the shadows baked into vertex colours, which is why I'm wondering how people dynamically shade objects when just using lightmaps etc
A lot of careful tuning and balancing went into the look of the game, to make sure we hit the visual cues that matter the most to our game. Reflections on car paint were definitely right at the top of our list, so we were happy to allocate some more resources to that, despite the fact it was quite costly. When rendering the reflection you can afford to take some shortcuts, because the image tends to be mapped and deformed onto highly curved surfaces, so things that would be very noticeable on a flat image, tend to get hidden somewhat. Two things that work really well is applying very aggressive culling as missing objects aren't that obvious, and lowering shader/texture detail.
One very effective technique for dynamic light shadowing is cascaded shadow maps. Multiple maps are rendered over the intersection of the camera frustum and the world (usually 2,3, or 4). When rendering the scene, the appropriate shadow map is picked based on distance from the camera. Since the farthest objects are also smallest on the screen, you can afford to drop the resolution of the shadow drastically for the far area.
There is nothing special about a dynamically lit character entering a statically shadowed world area. Provided that the dynamic shadow map is using the same light position, direction, and world geometry as the one used in the statically baked lightmap, the shadows on the character itself should always match. The trickier part is combining dynamic shadow that the object casts on the ground with the lightmap information, especially when the softness and overall quality of the two differ.
It's basically what I've implemented in my environment mapper. I actually have it setup as a seperate render engine with its own list of objects, so I can add only the essential stuff, and also assign a very short draw/culling distance to the 6 cameras.
Did you use cube mapping, or a faster less accurate method like dual paraboloid as Eric suggested?
Indeed I researched cascaded shadow maps too, and apparently they are popular on consoles. I decided against it as I batch a LOT of stuff to cut down on draw calls and I'm trying to keep things simple, so I'm not sure how useful it would be to me. But I might try it out as and when.
You know, I actually tried out what you described last night. Using the exact light position as Max, a dynamically shadow mapped car along with a static lightmapped level worked remarkably well. I assigned only the car to receive shadows, with a 512 map and very small shadowmap area to maximise resolution, and the shadows matched perfectly (ie a half shaded car when driving along the edge of a lightmapped shadow).
What I was enquiring about is a little different. I more meant fake shading the entire car if no dynamic shadow map is involved at all, by simply doing a manual lightmap lookup using the car's position.
This would be ideal if you really wanted to optimise performance by not rendering any of the geometry twice for a dynamic shadowmap, but you'd obviously lose some cool stuff like a half shadowed car, or tree shadows scrolling along the car in a foresty level etc
Oh and I agree with you, that's something I have found very tricky - having the level receive both dynamic shadows and having baked shadows. The dynamic shadows are really jarring as they simply look much worse than the softer baked shadows, that's why I decided on having only the car receive shadows as a happy medium
Cheers.
The lighting is not realistic and it was done to make it easier to avoid objects. But we have full dynamic shadows on the player vehicle, enemy vehicles and objects on the track.
To get it to run efficiently, I created custom shadow meshes for each object. These custom shadow meshes had 90% fewer polys than the main mesh. I set the shadow distance in the quality settings to a very short distance, under 100 units, to allow the shadows to maintain a decent sharpness even at a low setting. I use a shader on the shadow casters that renders them invisible but still casts a shadow. I turn off all self-shadowing.
Everything is done with forward rendering. One direct light casting dynamic shadows and up to four vertex lights which cast no shadow.
The environment can only receive dynamic cast shadows from the vehicles and objects on the track. The environments are lightmapped. Maps are baked in 3DS Max using VRay. I'm considering adding light probes but they would have to be subtle as the objects need to contrast clearly with the environment. Contrast is important in racing games to improve reaction times.
When we say mobile we are lumping in a huge range of devices and rendering capabilities. In the case of iOS, you may be supporting an iPhone 5/5s with a screen resolution of 640 x 1136 and you must support the iPad Mini 2 with a screen resolution of 2048 x 1536. Optimizing for the iPad Mini 2 is much harder than for the iPhone 5. Android is easier because you can say what exact devices you support or don't support.
Getting our game to run at a solid 30 fps on the iPad Mini 2 using the full screen resolution without sacrificing anything was challenging.
One more question, what exactly do you mean by vertex light? A dynamic light/shader that only calculates lighting per vertex rather than per pixel, like gouraud shading?
You know, even though I mentioned above creating lower poly geometry for reflections, it completely slipped my mind for shadow casting, but of course it makes perfect sense too.
Incidentally you're the first developer I've heard say it's easier to develop for Android. I thought iOS was considered easier as you know exactly what you're developing for, a fairly small handful of devices really, but the Android hardware is vast and highly variable.
By reducing the shadow distance to around 60 - 80 and not spreading it off into the horizon you can get away with some decent real-time shadows at low settings. You could even swap out to a blob shadow on an LOD if you notice any popping at far distances.
Environment lightmaps are 256 or 512 but that is mainly to reduce memory requirements. I also set them mostly to 2 bit PVRTC on iOS but if I notice any glitches I will bump them back to 4-bit.
I made all of the VFX in the game so glad you like them
Vertex lighting is as you described, lighting per vertex. It's very cheap to add in Unity and I use it for things like missles. I attach a vertex only light and as the missle flies past things it lights them up. It doesn't affect normal maps but it's moving so fast you can't tell. I also use vertex lights as fill lights in our garage and in parts of the game.
The shadow casting object also has more parts welded into a single mesh with a single shader so that makes it even faster. On the vehicles, the wheels are spinning so I can't weld them but for the shadowcaster object the wheels are welded to the body. This makes the shadowcaster draw call equal 1 instead of 3 or 4. Multiply by number of shadowcasting vehicles and you save around 10 shadowcaster calls. So, our total number of shadowcaster calls per frame is usually between 6 - 12. Total drawcalls is averaging 150 - 200.
It isn't easier or harder to do Android vs iOS, but with Android you can drop individual devices if you run into problems and don't want to deal with it. With iOS, you must support certain devices and can't simply choose not to support them.
I can get pretty fast directional lighting by not normalising the normal per pixel, and not having specular, but who wants 4 or 5 directional lights?
I would like something as you described, point lights for fires and particles, only lighting surrounding areas, and this sounds like a perfect use for fast vertex lighting. Especially if the light is moving, you don't need the fidelity of per pixel lighting I guess.
Good idea about shadow caster welding too, to save draw calls.
150 to 200 draw calls seems like a decent amount for mobiles (though obviously not for Metal). What framerate are you hitting on say, an iPhone 6? I've done fairly extensive benchmarks with my own engine, and I've been pleasantly surprised with the poly and pixel pushing power of modern phones, but draw calls do seem to be the bane of mobile performance.
I was hoping to keep it under 100 for my first mobile game, though for my 2.5d horror game, I've found it almost impossible to do that even with heavy batching, as I have DOF and dynamic shadows, so the scene needs to be rendered 3 times. Even though it's a smallish viewable area, there are a lot of moving objects, and it's difficult to keep it to just 30 objects or so on screen.
A quick question - is it possible to debug the shadow map in Unity? IE just show it as an overlay on screen?
The reason I ask, is I must be doing something wrong with my shadowmapper, or they must be doing something clever that I'm not. The reason being, is if I have a 512 shadowmap with a range of only say, 20% of the draw distance of a racer, the shadows look like a joke. Just a pixellated mess, not even close to the quality of shadows you have
They look fine in a small area like a room, but in a larger outdoor area I struggle to get shadows as good as yours even with a 2048. To get shadows similar to yours with a 512 I need to assign the shadow area to be absolutely tiny, like 2 or 3 car lengths.
Has your game already been released btw?
In Unity, you can set it to 60 fps whatever you want at the expense of responsiveness. It will also generate more heat which kills batteries and lowers the playtime of your game.
https://docs.unity3d.com/351/Documentation/Manual/iphone-Optimizing-MainLoop.html
Under 100 drawcalls is really dependent on the design of the game. In our case, we have a lot of depth in the scene plus all the other stuff going on so hitting 100 would be hard. If we had only one or two enemies on screen, roads with more bends so I could use some type of occlusion. fewer vfx, etc. then maybe. But we have a solid framerate with no stutter so I don't fixate on the absolute number of drawcalls.
I've never looked at the depth maps or shadow maps directly. I stick with the Unity settings and don't monkey around with them. I do use depth maps in shaders. So yes, you can debug and access any of the maps in Unity it just takes a bit of scripting. Here's a detailed explanation of the shadows in Unity: http://http//catlikecoding.com/unity/tutorials/rendering/part-7/
You can render the shadow cascade regions in a Unity scene view so you can visualize where the detail is going.
DOF is a killer for framerate. I love full screen image effects and color grading but it's just not viable for our game on mobile. I had some full screen image effects running on our UI but Unity broke render to texture on mobile so I had to abandon it. If we do a PC version I will be all over it
2- 3 car lengths actually sounds about right for a mobile game. Beyond that distance, I would swap to an LOD with a less detailed shader and a projector shadow. The projector texture I would make from a slightly angled down view of the vehicle to pick up some of the vehicle outline. Beyond a certain distance I turn shadows off completely because it's too expensive to just have two or three darker pixels under the car.
I would also be careful on how much of your environment is tagged to cast and receive shadow. The only thing that should be receiving shadow is the road directly under the vehicle and maybe some of the road shoulder. Break your road, shoulder and anything past the shoulder into separate pieces so you can tag them to receive or not receive shadows. You have to do this sort of art management because even with a 2048 shadow map you can waste the resolution on things the player can barely see.
The game is in soft-launch in a few countries. We're tweaking some UI things but the game is running great and ready for the next step. There's some business details and funding that is being worked out right now but the game is solid. We're also open to new investors so anyone feel free to contact ryan.blanchard@artcode.com and I can put them in touch with our CEO.
Phew, I'm glad you told me that. I'm not doing something wrong afterall, you obviously do just need a tiny shadowmapped area to get high quality shadows. I guess a lot of games do a good job of making the blend between shadowmapped and lightmapped/fake shadowed areas seamless, so it's hard to tell how big an area they're using.
Now I know that, I've realised you can have a near perfect player shadow with only a 512 actually, if you assign the area to be barely 2 car lengths.
Oh and I agree - that's exactly what I've done, just assigned the road and wall shader to receive shadows.
I'm new to GPU programming, and one of the frustrating things about assembly for me has been the lack of if/else conditions. It would be great to do something like if (pixel.z < 0.1) {check shadowMap} else {do nothing}.
Thanks for the link btw, it's an interesting read.
What do you mean by Unity broke RTT on mobile by the way? Do you mean they actually removed it on purpose, or did something that made it perform worse than before?
Yeah I thought you were targeting 30fps from the video. It certainly gives a lot more scope per frame. Sometimes I wish I wasn't such a framerate junkie, as it's pretty difficult to do everything you want in just 16.66ms. But I do love 60fps, so I'm targeting that for recent devices.
And yeah DOF is pretty brutal. Especially on devices with limited pixel power, it can almost halve the framerate even on fairly simple scenes.
It is essential to the look I'm going for though, as is bloom, so I've tried to get it running as fast as possible.
Sometimes I feel jealous of you Unity guys, as the Unity engine and workflow do look excellent. My engine might be less fancy, but it is fast that's for sure. I have a scene with thousands of moving particles, DOF and bloom running at 60fps on an iPhone 6.
Cheers.
The first thing I remove from a mobile shader file is any conditionals. Conditionals make it render twice and then the shader blends the two based on the condition.
When I worked in electronics I programmed in assembly on both CISC and RISC 8-bit and 16-bit microprocessors. A little 8-bit Pic microprocessor doesn't even have a MUL instruction.
The RTT issue has been fixed, you can read about it here: https://forum.unity3d.com/threads/anyone-know-what-this-error-means-m_size-k_reference_bit.388339/ I just wrote a shader that worked on each UI element instead of as a full screen image effect.
Unity is awesome but so are Unreal and Cryengine. I'm happy to use any modern engine that gives me access. The art and techniques of optimizations transfers to any engine.
And damn I didn't know that. I was interested in conditionals that can be used by targeting OpenGL ES 3.0, but now I know all they do is render twice, forget it, I thought it would actually improve performance.
Thanks again for the tips btw, they proved very useful. Upon reading that Unity shadow tutorial, I realised Unity creates a screenspace shadowmap in order to filter it before applying. I had a similar idea for soft shadows months ago when I created my shadowmapper (rendering the shadows in a seperate pass, then blurring the shadow buffer and applying it to the scene). It didn't occur to me to use the screenspace shadowmap as basically a lightmap though, used as a multiplier for the light values.
And the vertex lighting sounds really good, a cheap way to have multiple point lights.
Hopefully I can implement both of them this weekend
Cheers.
http://answers.unity3d.com/questions/442688/shader-if-else-performance.html
The specular highlights look crap, but the diffuse lighting looks surprisingly good. So it's very viable for moving point lights etc. And you can have around 12 moving point lights on a Note 2, with a decent polycount, running at 60fps - that's nuts.
Back on topic, do any of you guys know of a clever/creative way of achieving fast realtime soft shadows that doesn't involve simply sampling the shadowmap 4 or more times? Maybe it's unrealistic on mobile hardware, but I would love to have nice looking soft shadows, even if it meant reducing overall shadow resolution. Shadows are probably the hardest part of a 3D engine ime.