Looking for some basic help with foliage/trees/shading for 3ds Max/Unity

drysider

Hi everybody!

I love creating environment assets and foliage, and in the past I've never had to bother with vertex normals and shaders because I primarily work with unlit textures.
However, this time around I'm tackling shaded foliage assets and I'm looking for some simple direction. I'm using 3ds Max and these assets are intended for Unity.

Here's my work in progress tree. Definitely nowhere near finished but I've been mucking around with it testing how to tackle shading etc. Using SlideNormalThief I've copied normals from a sphere shape to my leaves, but I'm getting really ugly shading issues within Max. All backfaces of my leaves turn black, no matter what shape they are (totally flat cards, or curved), even with 2 sided on. Just looking for some guidance on how to avoid this issue within 3ds Max, and also within Unity.



Also got a super quick additional question - these trees can be chopped down in the project they're intended for, and the trunk is split into two pieces, the trunk and stump.  The texture seam isn't too big of an issue because I texture with 3d Coat, but there's a pretty obvious shading line where the stump and trunk meet because they're separate objects. Any way I can do this better to create a seamless tree, whether through modeling or through Unity?

Eric Chadwick

What lights do you have in the scene?
What shading mode for the viewport?
What material setup?
Which version of Max?

The view in Max will always be poorer than the view in Unity. I suggest exporting early and often. Use Unity as your viewport.

Your leaves texture looks bad, can you show the texture flat?

The stump seam can be hidden by averaging the vertex normals between trunk and stump. Select both, add an Edit Normals modifier, select one vertex pair, Average, repeat for all pairs. There are scripts to automate this.

Eric Chadwick

Also, turn on Backface Culling in Object Properties. This should help the black faces problem. You need to generate those faces manually (Detach, Flip) since you'll have to edit their normals too.

There's a Preference setting to turn it on at object creation, I always use this setting since game engines usually have it on too.

drysider

Averaging the normals on both trunk and stump worked perfectly, thank you! I figured it would be a vertex normal issue but didn't know where to start.

Somehow, as I've continued working on other assets, the leaves lighting problem seems to have fixed itself? I'm really not sure what I did, I don't think I changed anything. Maybe it was Max just bugging out or something? I have no lights in scene, working with default lights in viewport, just a basic Blinn with diffuse and opacity map. Max 2015.



Just so I'm understanding - to generate backfaces effectively, I need to clone the card and flip the normals of that clone?

Here's my current texture, it's nowhere near finished and I'm still trying to find an attractive leaf arrangement. I may even just scrap most of what I have since I'm not happy with the shape. The space above the leaves will be an additional leaf card to fill help fill out the foliage.

Eric Chadwick

It's usually better to use a separate texture for anything with transparency, for two reasons: 
1. Textures with an alpha channel are larger when compressed for use in a game. Often the file size is twice as large as without, for example http://wiki.polycount.com/wiki/DXT#DXT5.
2. Shaders with transparency are more expensive to render than shaders without. So it's better to not use the same shader on the trunk.

In 3ds Max 2015 there are two viewport modes... Realistic, and Shaded. Shaded is usually better for game work since it doesn't add fancy lighting effects like shadowing and ambient occlusion. You can turn those off. But Max also treats transpanency differently in the two modes, so try them out to see which is best. Anyhow, a game engine is always going to be better.

Your leaf arrangement is poor. It should look like an actual branch, with twigs in a branching pattern and a natural leaf arrangement. Often it's best to make a highpoly model and bake that down to a texture.

We have lots of resources here in case you haven't seen them...
http://wiki.polycount.com/wiki/Foliage

FourtyNights

From my experience, I've usually separated transparent and opaque materials to their own materials. In your case the trunk of the tree is opaque and leaves are transparent. But it also depends on the tranparency method in Unity, Cutout or Transparent.

For example, I had this one helmet I modeled, which had a transparent visor. I unwrapped the whole model to the one UV set, with one material. I got transparency sorting problems in Unity. So, as I separated the visor from the rest of the opaque model to a new material and modified its UVs to have more texel density, it started working.

EDIT: Yes, duplicating and flipping the normals of the transparent cards do the trick to have them "two sided". But remember also scale the cards away from their counterparts to avoid flickering if they're overlapped. Very small gap between the counterpart cards is good.

Eric Chadwick

I used to separate the front and back slightly, but realized the flicker was only because backface culling wasn't actually on!

With culling enabled, the front and back faces never flicker. They're facing separate directions. Unless you have some other sort of overlap problem, like two branches being co-planar.

We have some stuff on alpha sorting here, probably why the visor problem happened.
http://wiki.polycount.com/wiki/TransparencyMap#Sorting_Problems

drysider

Fantastic, thank you for the help and answers! The info about needing to clone the cards and flip them is great in particular, I had no idea that was standard since most environment assets I've made in the past haven't been intended for in-engine use. Will be great information when I work on other assets like grass etc etc. I've had a brief browse through the Polycount wiki foliage page but I'll refer back to it.

Thanks for the feedback on the leaves too, I actually do have a branch part of the texture, but it wasn't completed and didn't look too natural so I removed it for the time being. Work in progress etc. Since you recommend having the leaves texture be a separate material it'll give me a good chance to start over with a better more natural looking leaf formation and more space!