Hey people. Recently I started working on a small stylized meadow scene, inspired by the beautiful work of Helder Pinto, called Europa. It has a very painterly look, similar to BotW. I was able to achieve similar style / looking foliage in a fairly easy way and I thought I would share my experiences with you guys, regarding stylized foliage shading.
For teasing purposes, this is how my results looks currently:
As some of you may already know, edited vertex normals are often used on foliage meshes, to smooth the shading of the cards. If this isn't done, the rotation of the cards will make ugly, sudden changes in the shading. This is one of the things that needs to be fixed, but there is more.
Here is a tree mesh without edited vertex normals, to demonstrate the issue:
Not too bad, but we can clearly see those shading changes everywhere. Now here is a tree with edited vertex normals:
Despite the world normal pass looking smooth, we can still see some darker leaves. This is because the leaf planes has darker shading on the back face of them. This is a natural behavior, but it looks like we don't want it. Luckily, we can make both sides of the planes have the same lighting using a simple trick in the material. In Unreal, you can do this, and hook it up to the normal input:
We simply flip the normals on the back faces.
And the result will look like this:
You can play with the shadow bias and resolution settings to adjust the falloff of the shadows.
I did the same tricks with the grass in this scene:
Before:
After:
Grass world normals after the tricks applied:
Now these tricks might not be applicable in all cases, and may not look as good with realistic art style, but it definitely works very nicely with heavily stylized assets, for almost zero rendering cost and a little bit of work with the normals(which is actually automatic, because its done with a script.).
I hope that this breakdown will be useful for some people. Have a nice day!
Replies
One other thing that helps too is baking in some AO into the vertex colors and pump that into the Ambient Occlusion in unreal.
This really helps the model self shadow and it can be almost impossible to do AO the standard route with texture map when so many separate leaf planes, laying them out in their own unique UV space to make a lightmap would be very impractical.
Another trick is to use DFAO (Distance Field Ambient Occlusion) this works by making a shadow volume out of voxels, for each model. It can make foliage and just about everything, look really good, by shadowing the underside of a canopy and the inner parts of the tree.
You do have to be careful. It can be costly in game and it can take a while it calculates the distance fields, when loading a level for the first time or adjusting distance field settings. Tuning the DF on each model can be a bitch sometimes too, but it does looks pretty awesome...
One key thing to note about DFAO, is that the shadow volume is static and if you use anything like pixel depth offset to fake a bit of wind the distance field for that object won't move with it so you usually get some flickering. That's why I generally stick to the vertcolor AO bake method, it looks almost as good as DFAO but has less overhead and less glitches.
https://quixel.com/assets/shrkeiwi2
Example image from wiki:
The only difference is that my ones were only cards instead of these axis aligned crossed planes.
Blender have a built-in normal tranfer for it's own, and I've seen a result pretty similar to that one.
Also, take a look at this vertex control: https://www.youtube.com/watch?v=6BdzaK2z43s
Example usage:
@Maximum-Dev - In case of instanced static meshes, you could do it like this. Unfortunately the object radius has to be manually set. I didn't find a way to stabilize it.
The tree on the right is using your tech and the one on the left as just cast shadows disabled how can i reduce / remove all the small inner shadows when cast shadows is on ?
i've tried playing with bias but only putting it to something between 5 and 7 would make them vanish which is way to high for other objects.
Changing shadow resolution doesn't work also i'm using dynamic lighting !
Thanks in advance !
HI there, did you find a solution ? im facing a similar problem with my grass.
Edit - I see it has some... Maybe try changing the falloff of it.