Use Vertex Normals For Better LODs!
polycounter lvl 6
Recently I've come to love vertex normals and have fully adopted them into my workflow. And as of not too long ago I have come across another great benefit of vertex normals.
(Sorry if most of you already know this, I'm mostly out of the loop and learn these things independently.)
(Sorry if most of you already know this, I'm mostly out of the loop and learn these things independently.)
Vertex normals can be tricky to handle, and if you use them wrong you'll end up getting serious artifacts, baking issues and overall problems but if you know how to use them well, they can be really helpful.
I'm going to give an example I'm currently working on, the UMP 45.
Here's my lowpoly, as you can see the shading looks mostly flat, and you can't see many details.
Now with the wireframe enabled, you can see that there are in fact a few bevels and bumps. What I've done is I've set the shading to be very flat, so that each side (left, down, right) has all the geometry normals facing a single way. Rather than looking like a cylinder, the shading makes it look more like a chamfered square.
This bakes without a problem, that's the point of a normal map.
Now looking at the baked normal map, at the bevel for example, does this remind you of anything? Maybe hardsurfaces that you paint within Substance Painter?
Now maybe you can guess why this is good for LODs. Now I'm going to remove this bevel from my lowpoly entirely as well as a few other details, by mostly using Ctrl+Backspace.
Here, I have the original lowpoly on top, and the reduced lowpoly on the bottom. I've removed the all three of these details from the sillhouete, and since the vertex normals were flat in the first place, they're still flat now and the exact same normal map appears exactly the same.
(There are still two circles of vertices that were holes that I've kept, because I had a UV split there, there rest was all a single UV & smoothing group so I could remove the geo without a problem).
In summary, a normal map will try to make your lowpoly model appear the same way as the highpoly. Vertex normals also influence what your final normal looks like, and if you have ugly vertex normals then the normal map will try to compensate for this. If you suddenly remove that goemetry, then the normal map will still still try to compensate for the original vertex normals even if they're no longer there. This method just basically removes the normals in the first place, meaning when you remove them manually the final normal appears exactly the same.
That's just my 2 cents worth, I thought it's worth sharing so maybe it'll help someone out. I found this very useful whilst making LODs for my previous weapon, even though I didn't really know about this, I just had made normals flat in a bunch of areas without knowing 
. I seem to remember CryEngine's LOD tools make a normal map automatically for each LOD, though I think Unreal lacks that, and also that's automatic, sometimes you need or want to do things by hand within 3ds max.
. I seem to remember CryEngine's LOD tools make a normal map automatically for each LOD, though I think Unreal lacks that, and also that's automatic, sometimes you need or want to do things by hand within 3ds max.
Replies
Chiming in cause I had a similar idea but different workflow to use toolbag baker better and more efficiently.
1, make basemesh as detailed as I want my lowpoly to be. Unwrap the basemesh (with supporting geometry and all), and bake object space normals in toolbag. Because the basemesh has all that supporting geometry, there are very few issues with the generated cage and I have more control over the normals with the painty thing. Texture the basemesh.
2, duplicate basemesh, call it lowpoly, remove all the supporting geometry easily because I did them all with modifiers, retriangulate to solve the weird UV issues that pop up, then use handplane to transfer the objectspace normalmap and textures from the UV'd basemesh onto the lowpoly. Because there's nearly no difference between them, there's no issues in that bake, and no cage needed.
LOD1 can then be a duplicate of lowpoly that I remove a bunch of edges from, fill in holes, retriangulate, transfer object space normals again.
After texturing a 50k tri basemesh, I spent a total of 45 minutes making 3 LODs with perfect bakes and representations of my textures:
Lowpoly: 13k tris, retriangulated derivative of basemesh, exact same external geometry as basemesh, so no cage needed.
LOD1: 3k tris, retriangulated derivative of lowpoly, very little remodeling, textures and normals 'baked' from lowpoly. No cage needed.
LOD2: 300 tris, almost complete remodel, textures and normals 'baked' from LOD1. Cage needed!
I don't mean to hijack your thread, but I thought your thing was super neato, so I hope it's OK to share something else that's sort of neato in your thread.
I haven't had the chance to work with object space normals before so I don't know much about it, is it the same as baking vertex normals into a texture map?
Whilst flat areas are probably going to end up fine with turbosmooth, any curved areas will have different smoothing once you apply almost any modifier. Once the baked normal is applied to the lowpoly when baked with a higherpoly it'll end up mismatching, even if it's pretty slight, especially if you have stuff like sharper triangular variations in smoothing which are common.
The only way I could find within max to overcome different smoothing is to use the subdivide modifier, it *mostly* keeps the smoothing exactly the same whilst adding in lots of geo, sometimes doesn't. It's also just annoying to have two work with two sets of lowpolies.