Home Technical Talk

Common shadow setups for mobile games?

vertex
Offline / Send Message
Raffles vertex
Hello,

I was really happy recently when I got dynamic shadow mapping working in my mobile engine. It works perfectly for my 2.5d horror game as the viewable area is smallish, so the resolution of the shadow map is acceptable.

However - I'm also prototyping a mobile racer, and unfortunately my hopes for fully dynamic shadows were quashed in an instant ;)

Even with a 2048 shadow map covering approximately half of the viewable area, the shadows are far too low res.

So I was wondering what a common shadow setup is for mobile games or even console games? Presumably lightmapping for most of the level, and dynamic shadow mapping in as small an area as possible near the camera, for the player and other nearby dynamic objects?

Incidentally I checked out GTA5 recently for the first time in a while, and I was impressed with how good the shadows are. There appear to be dynamic shadows on almost everything, and they are good quality too, better than most open world games. Amazing given the sheer scope of the environment and the age of the hardware it's running on.

Cheers.

Replies

  • Eric Chadwick
    Most mobile games avoid dynamic shadowing altogether as being too expensive. Blob shadows instead (alpha blended decals).

    @RyanB has been posting some great optimization tips in the Unity section. For example:
    http://polycount.com/discussion/182384/lighting-for-mobile-in-unity-optimization
  • Raffles
    Offline / Send Message
    Raffles vertex
    Thanks for the link @Eric Chadwick I'll check it out in a minute.

    I think you're right about the shadows. I checked out some mobile games, and even Asphalt for example appears to have just lightmaps with a fake shadow raycast into the floor.

    I'm somewhat surprised though. On my mobile game I have fairly complex scenes with around 100,000 polys being rendered, with dynamic shadows being cast and received by everything, and a 1024 shadowmap running at 30fps on a Note 2. Crucially though, with hard shadows. Doing the shadowmap sampling and calculations once per pixel is doable, 16 times not so much.

    I also played Real Racing 3 recently, and I was surprised to see real time environment mapping, running well even on a 4 year old phone.

    That's something else I also got running recently, and unless there's a cheaper way I don't know about, it's expensive. You have to render the scene 6 times, once for each side of a cube map. Even if you use 128 textures, the extra draw calls and polys rendered is not exactly trivial. The only thing I can think of is they build some very low poly batched geometry specifically for reflections, and render that to the cubemap instead of the standard models.
  • Eric Chadwick
    Yes, that is very likely. Old school method... only render-to-texture the most visible meshes, discard everything small, use simpler shaders, etc.

    You could also look into dual-paraboloid reflection mapping. Then you're only rendering two views. Up/down or back/front. There's a noticeable seam sometimes, but usually not that big a deal on something distorted like a car body. They might even be rendering only one view (facing back at the players camera).
  • Raffles
    Offline / Send Message
    Raffles vertex
    Thanks for the tip Eric, that actually sounds very promising. Just like you said, common uses for real time reflections include things like cars, which are not only a small area on screen, but pretty distorted. I'd imagine this method would work fine unless you were inspecting closely.

    Incidentally I had a look at Ridge Racer 6 the other day for the first time in years, and they had real time reflections yet no dynamic shadows. Again I was pretty surprised, as my preconceived notion was that it would be far more expensive than dynamic shadows.

    Cheers.
  • Raffles
    Offline / Send Message
    Raffles vertex
    @Eric Chadwick One more thing - do you happen to know how most people shade dynamic objects when they fall into lightmapped areas etc? I've played a couple of mobile games now that have obvious lightmaps and fake shadows for the player, yet the player does get shaded in baked shadow areas.

    There's a couple of possibilities I can think of.

    1 - Just raycast towards the main shadowcasting light, and if you hit something, shade the character. Obviously a bit crap and limited, but should work.

    2 - A better option would be to raycast down, fetch the triangle the ray hits, calculate the UVs of the intersection, then do a manual lightmap lookup to get the exact value of the lightmap that the player is on.

    You could then feed this value to the shader, and add just one line of code to the fragment shader, something like

    mul targetReg targetReg dynamicShadeReg

    Oh and btw thanks to @Swordslayer I got a pretty good, fast shadow setup working for my mobile racer. Exported vertex colour lighting instead of lightmapping, it's just so fast and easy. Then a dynamic shadowmapper just for the car, so world geometry shadows get cast onto the car.

    Unfortunately though I'm having trouble getting the in game shadowcasting to match the shadows baked into vertex colours, which is why I'm wondering how people dynamically shade objects when just using lightmaps etc :)
  • kwyjibo
    Offline / Send Message
    kwyjibo polycounter lvl 3
    Raffles said:
    do you happen to know how most people shade dynamic objects when they fall into lightmapped areas etc? I've played a couple of mobile games now that have obvious lightmaps and fake shadows for the player, yet the player does get shaded in baked shadow areas.

    They are most likely using light probes.
  • ileben
    Offline / Send Message
    ileben null
    Raffles said:
    [...]
    I also played Real Racing 3 recently, and I was surprised to see real time environment mapping, running well even on a 4 year old phone.

    That's something else I also got running recently, and unless there's a cheaper way I don't know about, it's expensive. You have to render the scene 6 times, once for each side of a cube map. Even if you use 128 textures, the extra draw calls and polys rendered is not exactly trivial. The only thing I can think of is they build some very low poly batched geometry specifically for reflections, and render that to the cubemap instead of the standard models.
    Thanks, I'm glad you enjoyed the look of Real Racing 3! (RR graphics dev here)
    A lot of careful tuning and balancing went into the look of the game, to make sure we hit the visual cues that matter the most to our game. Reflections on car paint were definitely right at the top of our list, so we were happy to allocate some more resources to that, despite the fact it was quite costly. When rendering the reflection you can afford to take some shortcuts, because the image tends to be mapped and deformed onto highly curved surfaces, so things that would be very noticeable on a flat image, tend to get hidden somewhat. Two things that work really well is applying very aggressive culling as missing objects aren't that obvious, and lowering shader/texture detail.

    One very effective technique for dynamic light shadowing is cascaded shadow maps. Multiple maps are rendered over the intersection of the camera frustum and the world (usually 2,3, or 4). When rendering the scene, the appropriate shadow map is picked based on distance from the camera. Since the farthest objects are also smallest on the screen, you can afford to drop the resolution of the shadow drastically for the far area.

    There is nothing special about a dynamically lit character entering a statically shadowed world area. Provided that the dynamic shadow map is using the same light position, direction, and world geometry as the one used in the statically baked lightmap, the shadows on the character itself should always match. The trickier part is combining dynamic shadow that the object casts on the ground with the lightmap information, especially when the softness and overall quality of the two differ.

  • Raffles
    Offline / Send Message
    Raffles vertex
    @kwyjibo OK thanks, I'll look into that, it's something I haven't implemented.
  • Raffles
    Offline / Send Message
    Raffles vertex
    ileben said:
    Raffles said:
    [...]
    I also played Real Racing 3 recently, and I was surprised to see real time environment mapping, running well even on a 4 year old phone.

    That's something else I also got running recently, and unless there's a cheaper way I don't know about, it's expensive. You have to render the scene 6 times, once for each side of a cube map. Even if you use 128 textures, the extra draw calls and polys rendered is not exactly trivial. The only thing I can think of is they build some very low poly batched geometry specifically for reflections, and render that to the cubemap instead of the standard models.
    Thanks, I'm glad you enjoyed the look of Real Racing 3! (RR graphics dev here)
    A lot of careful tuning and balancing went into the look of the game, to make sure we hit the visual cues that matter the most to our game. Reflections on car paint were definitely right at the top of our list, so we were happy to allocate some more resources to that, despite the fact it was quite costly. When rendering the reflection you can afford to take some shortcuts, because the image tends to be mapped and deformed onto highly curved surfaces, so things that would be very noticeable on a flat image, tend to get hidden somewhat. Two things that work really well is applying very aggressive culling as missing objects aren't that obvious, and lowering shader/texture detail.

    One very effective technique for dynamic light shadowing is cascaded shadow maps. Multiple maps are rendered over the intersection of the camera frustum and the world (usually 2,3, or 4). When rendering the scene, the appropriate shadow map is picked based on distance from the camera. Since the farthest objects are also smallest on the screen, you can afford to drop the resolution of the shadow drastically for the far area.

    There is nothing special about a dynamically lit character entering a statically shadowed world area. Provided that the dynamic shadow map is using the same light position, direction, and world geometry as the one used in the statically baked lightmap, the shadows on the character itself should always match. The trickier part is combining dynamic shadow that the object casts on the ground with the lightmap information, especially when the softness and overall quality of the two differ.

    Thanks for the reply and info. Indeed you did a great job with the graphics, especially given that it came out a few years ago and mobile technology moves at a frightening rate. I did notice the very aggressive culling, including a nice alpha fade in so as not to pop in too much ;)

    It's basically what I've implemented in my environment mapper. I actually have it setup as a seperate render engine with its own list of objects, so I can add only the essential stuff, and also assign a very short draw/culling distance to the 6 cameras.

    Did you use cube mapping, or a faster less accurate method like dual paraboloid as Eric suggested?

    Indeed I researched cascaded shadow maps too, and apparently they are popular on consoles. I decided against it as I batch a LOT of stuff to cut down on draw calls and I'm trying to keep things simple, so I'm not sure how useful it would be to me. But I might try it out as and when.

    You know, I actually tried out what you described last night. Using the exact light position as Max, a dynamically shadow mapped car along with a static lightmapped level worked remarkably well. I assigned only the car to receive shadows, with a 512 map and very small shadowmap area to maximise resolution, and the shadows matched perfectly (ie a half shaded car when driving along the edge of a lightmapped shadow).

    What I was enquiring about is a little different. I more meant fake shading the entire car if no dynamic shadow map is involved at all, by simply doing a manual lightmap lookup using the car's position.

    This would be ideal if you really wanted to optimise performance by not rendering any of the geometry twice for a dynamic shadowmap, but you'd obviously lose some cool stuff like a half shadowed car, or tree shadows scrolling along the car in a foresty level etc :)

    Oh and I agree with you, that's something I have found very tricky - having the level receive both dynamic shadows and having baked shadows. The dynamic shadows are really jarring as they simply look much worse than the softer baked shadows, that's why I decided on having only the car receive shadows as a happy medium ;)

    Cheers.
  • RyanB
    Offline / Send Message
    RyanB Polycount Sponsor
    I'm working on Speed Freeks, a Warhammer mobile game.   If you watch the soft-launch trailer, from 9 seconds to about 13 seconds , you can get a pretty good idea of the overall look we achieved. 


    The lighting is not realistic and it was done to make it easier to avoid objects.  But we have full dynamic shadows on the player vehicle, enemy vehicles and objects on the track. 

    To get it to run efficiently, I created custom shadow meshes for each object.  These custom shadow meshes had 90% fewer polys than the main mesh.  I set the shadow distance in the quality settings to a very short distance, under 100 units, to allow the shadows to maintain a decent sharpness even at a low setting.  I use a shader on the shadow casters that renders them invisible but still casts a shadow.  I turn off all self-shadowing.

    Everything is done with forward rendering.  One direct light casting dynamic shadows and up to four vertex lights which cast no shadow.

    The environment can only receive dynamic cast shadows from the vehicles and objects on the track.  The environments are lightmapped.  Maps are baked in 3DS Max using VRay.  I'm considering adding light probes but they would have to be subtle as the objects need to contrast clearly with the environment.  Contrast is important in racing games to improve reaction times.

    When we say mobile we are lumping in a huge range of devices and rendering capabilities.  In the case of iOS, you may be supporting an iPhone 5/5s with a screen resolution of 640 x 1136 and you must support the iPad Mini 2 with a screen resolution of 2048 x 1536.  Optimizing for the iPad Mini 2 is much harder than for the iPhone 5.   Android is easier because you can say what exact devices you support or don't support.

    Getting our game to run at a solid 30 fps on the iPad Mini 2 using the full screen resolution without sacrificing anything was challenging.



  • Raffles
    Offline / Send Message
    Raffles vertex
    Thanks for the info @RyanB, that's helpful. Your shadow setup looks good, they look nice and clean. NIce particles and explosions too. What engine are you using? And also, are you using a high enough resolution shadowmap to use hard shadows, or lower res with soft shadows? On an average phone, what kind of dent to the framerate do your dynamic shadows give?

    One more question, what exactly do you mean by vertex light? A dynamic light/shader that only calculates lighting per vertex rather than per pixel, like gouraud shading?

    You know, even though I mentioned above creating lower poly geometry for reflections, it completely slipped my mind for shadow casting, but of course it makes perfect sense too.

    Incidentally you're the first developer I've heard say it's easier to develop for Android. I thought iOS was considered easier as you know exactly what you're developing for, a fairly small handful of devices really, but the Android hardware is vast and highly variable.
  • RyanB
    Offline / Send Message
    RyanB Polycount Sponsor
    We're using Unity.  [Edit] Checked our settings and we are using 60 for shadow distance and Medium quality.  That would make it 1024. [/Edit] 

    By reducing the shadow distance to around 60 - 80 and not spreading it off into the horizon you can get away with some decent real-time shadows at low settings.  You could even swap out to a blob shadow on an LOD if you notice any popping at far distances. 

    Environment lightmaps are 256 or 512 but that is mainly to reduce memory requirements.  I also set them mostly to 2 bit PVRTC on iOS but if I notice any glitches I will bump them back to 4-bit.

    I made all of the VFX in the game so glad you like them :) .  That's a whole other optimization discussion. 

    Vertex lighting is as you described, lighting per vertex.  It's very cheap to add in Unity and I use it for things like missles.  I attach a vertex only light and as the missle flies past things it lights them up.  It doesn't affect normal maps but it's moving so fast you can't tell.  I also use vertex lights as fill lights in our garage and in parts of the game.

    The shadow casting object also has more parts welded into a single mesh with a single shader so that makes it even faster.  On the vehicles, the wheels are spinning so I can't weld them but for the shadowcaster object the wheels are welded to the body.  This makes the shadowcaster draw call equal 1 instead of 3 or 4.  Multiply by number of shadowcasting vehicles and you save around 10 shadowcaster calls.  So, our total number of shadowcaster calls per frame is usually between 6 - 12.  Total drawcalls is averaging 150 - 200.

    It isn't easier or harder to do Android vs iOS, but with Android you can drop individual devices if you run into problems and don't want to deal with it.  With iOS, you must support certain devices and can't simply choose not to support them.


  • Raffles
    Offline / Send Message
    Raffles vertex
    Hey @RyanB, ah thanks for the tip, that's actually very useful about the vertex lighting. I'll have to implement something similar.

    I can get pretty fast directional lighting by not normalising the normal per pixel, and not having specular, but who wants 4 or 5 directional lights? :D

    I would like something as you described, point lights for fires and particles, only lighting surrounding areas, and this sounds like a perfect use for fast vertex lighting. Especially if the light is moving, you don't need the fidelity of per pixel lighting I guess.

    Good idea about shadow caster welding too, to save draw calls.

    150 to 200 draw calls seems like a decent amount for mobiles (though obviously not for Metal). What framerate are you hitting on say, an iPhone 6? I've done fairly extensive benchmarks with my own engine, and I've been pleasantly surprised with the poly and pixel pushing power of modern phones, but draw calls do seem to be the bane of mobile performance.

    I was hoping to keep it under 100 for my first mobile game, though for my 2.5d horror game, I've found it almost impossible to do that even with heavy batching, as I have DOF and dynamic shadows, so the scene needs to be rendered 3 times. Even though it's a smallish viewable area, there are a lot of moving objects, and it's difficult to keep it to just 30 objects or so on screen.

    A quick question - is it possible to debug the shadow map in Unity? IE just show it as an overlay on screen?

    The reason I ask, is I must be doing something wrong with my shadowmapper, or they must be doing something clever that I'm not. The reason being, is if I have a 512 shadowmap with a range of only say, 20% of the draw distance of a racer, the shadows look like a joke. Just a pixellated mess, not even close to the quality of shadows you have :)

    They look fine in a small area like a room, but in a larger outdoor area I struggle to get shadows as good as yours even with a 2048. To get shadows similar to yours with a 512 I need to assign the shadow area to be absolutely tiny, like 2 or 3 car lengths.

    Has your game already been released btw?
  • RyanB
    Offline / Send Message
    RyanB Polycount Sponsor
    We chose to leave the rendering loop at 30 fps (default for Unity on iOS).  We are at a solid 30 fps on both cpu and gpu on the iPhone 6 and that is with tons of stuff on screen.  In the quieter moments of the game, we get around 100 drawcalls (geometry and transparent combined) and the game still renders at 30 fps but it becomes more responsive.  Even on the iPad Mini 2 we are at 30 fps.

    In Unity, you can set it to 60 fps whatever you want at the expense of responsiveness.  It will also generate more heat which kills batteries and lowers the playtime of your game. 
    https://docs.unity3d.com/351/Documentation/Manual/iphone-Optimizing-MainLoop.html

    Under 100 drawcalls is really dependent on the design of the game.  In our case, we have a lot of depth in the scene plus all the other stuff going on so hitting 100 would be hard.  If we had only one or two enemies on screen, roads with more bends so I could use some type of occlusion. fewer vfx, etc. then maybe.  But we have a solid framerate with no stutter so I don't fixate on the absolute number of drawcalls. 

    I've never looked at the depth maps or shadow maps directly.  I stick with the Unity settings and don't monkey around with them.  I do use depth maps in shaders.  So yes, you can debug and access any of the maps in Unity it just takes a bit of scripting.  Here's a detailed explanation of the shadows in Unity: http://http//catlikecoding.com/unity/tutorials/rendering/part-7/

    You can render the shadow cascade regions in a Unity scene view so you can visualize where the detail is going. 

    DOF is a killer for framerate.  I love full screen image effects and color grading but it's just not viable for our game on mobile.  I had some full screen image effects running on our UI but Unity broke render to texture on mobile so I had to abandon it.  If we do a PC version I will be all over it :)

    2- 3 car lengths actually sounds about right for a mobile game.  Beyond that distance, I would swap to an LOD with a less detailed shader and a projector shadow.  The projector texture I would make from a slightly angled down view of the vehicle to pick up some of the vehicle outline.  Beyond a certain distance I turn shadows off completely because it's too expensive to just have two or three darker pixels under the car. 

    I would also be careful on how much of your environment is tagged to cast and receive shadow.  The only thing that should be receiving shadow is the road directly under the vehicle and maybe some of the road shoulder.  Break your road, shoulder and anything past the shoulder into separate pieces so you can tag them to receive or not receive shadows.  You have to do this sort of art management because even with a 2048 shadow map you can waste the resolution on things the player can barely see.

    The game is in soft-launch in a few countries.  We're tweaking some UI things but the game is running great and ready for the next step.  There's some business details and funding that is being worked out right now but the game is solid.  We're also open to new investors so anyone feel free to contact [email protected] and I can put them in touch with our CEO.

  • Raffles
    Offline / Send Message
    Raffles vertex
    Hey @Ryan,

    Phew, I'm glad you told me that. I'm not doing something wrong afterall, you obviously do just need a tiny shadowmapped area to get high quality shadows. I guess a lot of games do a good job of making the blend between shadowmapped and lightmapped/fake shadowed areas seamless, so it's hard to tell how big an area they're using.

    Now I know that, I've realised you can have a near perfect player shadow with only a 512 actually, if you assign the area to be barely 2 car lengths.

    Oh and I agree - that's exactly what I've done, just assigned the road and wall shader to receive shadows.

    I'm new to GPU programming, and one of the frustrating things about assembly for me has been the lack of if/else conditions. It would be great to do something like if (pixel.z < 0.1) {check shadowMap} else {do nothing}.

    Thanks for the link btw, it's an interesting read.

    What do you mean by Unity broke RTT on mobile by the way? Do you mean they actually removed it on purpose, or did something that made it perform worse than before?

    Yeah I thought you were targeting 30fps from the video. It certainly gives a lot more scope per frame. Sometimes I wish I wasn't such a framerate junkie, as it's pretty difficult to do everything you want in just 16.66ms. But I do love 60fps, so I'm targeting that for recent devices.

    And yeah DOF is pretty brutal. Especially on devices with limited pixel power, it can almost halve the framerate even on fairly simple scenes.

    It is essential to the look I'm going for though, as is bloom, so I've tried to get it running as fast as possible.

    Sometimes I feel jealous of you Unity guys, as the Unity engine and workflow do look excellent. My engine might be less fancy, but it is fast that's for sure. I have a scene with thousands of moving particles, DOF and bloom running at 60fps on an iPhone 6.

    Cheers.

  • RyanB
    Offline / Send Message
    RyanB Polycount Sponsor
    I don't use ARB assembly language directly.  My workflow is generally prototype in Shader Forge then optimize the Cg/HLSL .shader files manually.  After I optimize those, I check the instruction counts by compiling in Unity.  I've tried optimizations I've read about in fairly recent papers and the compiler almost always has already taken care of it.  For example, I tried using Mult then Add (MAD) vs AddMul and the compiler gave the same result. 

    The first thing I remove from a mobile shader file is any conditionals.  Conditionals make it render twice and then the shader blends the two based on the condition. 

    When I worked in electronics I programmed in assembly on both CISC and RISC 8-bit and 16-bit microprocessors.  A little 8-bit Pic microprocessor doesn't even have a MUL instruction. :o

    The RTT issue has been fixed, you can read about it here: https://forum.unity3d.com/threads/anyone-know-what-this-error-means-m_size-k_reference_bit.388339/   I just wrote a shader that worked on each UI element instead of as a full screen image effect.

    Unity is awesome but so are Unreal and Cryengine.  I'm happy to use any modern engine that gives me access.  The art and techniques of optimizations transfers to any engine. 
  • Raffles
    Offline / Send Message
    Raffles vertex
    Wow, not even mul. I guess I should count my blessings I not only have mul, but also SGE and SLT, which are the closest thing to conditionals, and essential for shadow mapping etc.

    And damn I didn't know that. I was interested in conditionals that can be used by targeting OpenGL ES 3.0, but now I know all they do is render twice, forget it, I thought it would actually improve performance.

    Thanks again for the tips btw, they proved very useful. Upon reading that Unity shadow tutorial, I realised Unity creates a screenspace shadowmap in order to filter it before applying. I had a similar idea for soft shadows months ago when I created my shadowmapper (rendering the shadows in a seperate pass, then blurring the shadow buffer and applying it to the scene). It didn't occur to me to use the screenspace shadowmap as basically a lightmap though, used as a multiplier for the light values.

    And the vertex lighting sounds really good, a cheap way to have multiple point lights.

    Hopefully I can implement both of them this weekend ;)

    Cheers.
  • RyanB
    Offline / Send Message
    RyanB Polycount Sponsor
    There's a ton to learn about shaders but luckily there's also a ton of info online.  The slowdown from conditionals comes up a lot.  The hardware has gotten better but it's still usually bad.   https://www.opengl.org/discussion_boards/showthread.php/181028-conditionals-in-shaders
    http://answers.unity3d.com/questions/442688/shader-if-else-performance.html



  • Raffles
    Offline / Send Message
    Raffles vertex
    I'm glad you told me about the vertex lights, I've added support for directional, point, and spot vertex lights. Surprisingly easy and simple to setup. All the calculations are the same of course, you just do it per vertex, then pass variables for the diffuse and specular from vertex shader to fragment shader. They just get added on to the total light variable as if they were pixel lights.

    The specular highlights look crap, but the diffuse lighting looks surprisingly good. So it's very viable for moving point lights etc. And you can have around 12 moving point lights on a Note 2, with a decent polycount, running at 60fps - that's nuts.

    Back on topic, do any of you guys know of a clever/creative way of achieving fast realtime soft shadows that doesn't involve simply sampling the shadowmap 4 or more times? Maybe it's unrealistic on mobile hardware, but I would love to have nice looking soft shadows, even if it meant reducing overall shadow resolution. Shadows are probably the hardest part of a 3D engine ime.
Sign In or Register to comment.