How do I get soft normal foliage like this?

monagart

I'm currently working on a project for my uni course and have been trying to get trees and bushes with a certain effect. It's pretty much the same as can be seen in this game:

https://www.insidedecay.com/maquette

I am familiar with softening normals, or transferring them from a sphere on to a tree, but it doesn't quite get the exact effect. In these examples the normals of the leafs/alpha cards blend perfectly with the base of the tree, and the silhouette is perfectly coloured and softly shaded. I have experimented in engine and can't seem to get the same effect as the back of the alpha cards will have different normals, the foliage with cast shadows on each other, or the mesh won't cast a shadow on the ground. I'm working in unreal so not sure if there is a certain way to do it here.

This is the closet I've managed to get to the effect:




As you can see it seems to work on the right bush, but as soon as I apply the opacity maps the normals become black on one side, even with a two sided material or two sided foliage setting. I can't seem to figure this out so thought I'd ask and see

Eric Chadwick

Don't use a double-sided shader, duplicate and flip the polygons manually before bending the normals.

More about why:
http://wiki.polycount.com/wiki/Foliage#Vertex_Normals

Obscura

This thread discusses the same issue as well as some others:
https://polycount.com/discussion/209623/smooth-foliage-like-in-breath-of-the-wild-europa-by-helder-pinto-mini-tutorial#latest

What you are looking for against the black back faces is this:

monagart

Thank you so much @Eric Chadwick and @Obscura

I done a very quick test in engine trying some of this out and it seems to be working well. I'm going to make some proper meshes and try the same thing. Still need to add in some colour variation here and fix some shading but thanks for the help! 

monagart

So I feel like I'm almost there, but still having some issues with this. I'm not sure why it's started happening now, but the foliage alphas are casting harsh shadows on to the rest of the mesh. Weirdly enough the shadows look great from afar, but become a problem when close.

Here is is from afar. It's pretty much the EXACT look I want for the trees.



Here it is when you get closer:



And what they look like right at them:



Anyone out there know how to possibly fix this?

I have one are light and one directional light, both on moveable right now but the issue persists even when baked on static or stationery.

Shadow bias is at 1, and shadow sharpness has also been tweaked (but doesn't change much).

Eric Chadwick

Turn off self-shadowing?

Eric Chadwick

Looking at your reference, they don't self-shadow at all.

Obscura

Yep, usually self shadowing is disabled, or you can play with the bias parameter of the shadow to make the contact softer.

monagart

Hey, I actually ended up figuring this out, so thought I'd update this thread in case anyone else is ever wondering how to do this so they don't become stuck like me. Here is what I finally got:



I'm still fairly inexperienced with Unreal so the things I've done may not be the best or most efficient way, but it's working for me right now. Here is what I done:

1- Firstly I took my tree mesh in maya and used a dome to transfer the normals onto my tree mesh. I used mesh/transfer attributes in maya to transfer the vertex normals. Make sure that in the options for it you only have vertex normals selected. I like the explanation here: https://www.patreon.com/posts/stylized-tree-25192376, but I used a dome instead of a sphere.

2- In Unreal I set up my material as two sided, and plugged this into my material's normals to soften the backfaces.



3- If you are baking your skylight I recommend to deselect "cast shadow" as it was casting shadows from the foliage cards on to the middle of my tree mesh. You can also turn off "cast shadow" under the mesh details, but I wanted it to cast a shadow on other things so I had to try some other stuff out.

4- I went to my directional light and played around with the shadow bias which really softened the shadows on the foliage. This does affect the shadows for everything else at the same time, but a way to work around this is to use lighting channels (explained here).

5-  I also played around with "Num Dynamic Shadow Cascades", but be careful with this as it again affects all of the meshes. This basically makes the shadows lower resolution the further the camera is from them. In my case I wanted a really low resolution of shadows for the trees, so it worked for me. You could also use lighting channels to work around it.

And that is what I've managed to do so far! Thanks @Obscura and @Eric Chadwick for helping me

Panupat

I liked your last one more than your reference! That little dark gradient revealing leaf shapes you have totally sold me.

monagart

@Panupat thank you! Messing about in Unreal has made some nice effects