Editing Vertex Normals in the Industry?

Geosmith

Hello.

Currently I'm just a hobbyist planning to look for a job in 3D sometime soon. I like to be as independent as I can, so In my time in 3D rather than looking for an answer as to what's the norm or what someone *should* do in a situation, I find my own solutions that I feel have the best results. 

The one thing that I do that worries me the most is modifying vertex normals. I've been doing it for quite a while for a bunch of reasons. One reason is to improve overall smoothing of the mesh instead of using support loops or chamfers. I can remove any nasty sharp triangles and make any kind of smoothing much softer and plain for as clean a normal map as I can make. This also means I can reduce the amount of hard edges as long as the UVs allow it reducing overall vertex counts. Later down the pipeline, modified vertex normals can allow me to reduce areas of geometry for LODs without changing the normals at all like I've shared here: http://polycount.com/discussion/191056/use-vertex-normals-for-better-lods

I'm sure there's fast ways of recreating a normal map specifically for a lower LOD mesh, something CryEngine has tools for that (if I remember right), but I'm mostly a newbie in this area and my method works well for me and doesn't need any rebaking or multiple sets of normal maps. 

But there's also negatives when you're new to vertex normals, at least for 3DS Max. They're very fragile and will break with almost any modification to the mesh, most modifiers will destroy them and from what I've seen they don't have a tonne of support in some software. It's for some reason impossible to import both edited normals and smoothing groups using FBX for 3DS Max (have to use OBJ, far slower). But since I've been using them for quite some time I have no issues working with them and have scripts to make it much easier and faster to work with them. In plenty of cases it's easier and faster to modify normals rather than it would be to add support loops or mess with anything else. 


What worries me is how much the industry is accepting of this. This is nothing but a plus for me, but I can imagine when working in a group of artists it could be an obstacle for someone else. How often do jobs limit people to doing only one part of creating an asset (for example having different people for each stage of an asset like modelling, highpoly, texturing) where it would mean other people would have to work with my assets and could run into issues? I constantly see something along the lines of everything should always be set up and ready for someone else to come along and take over (e.g no photoshop normal map adjustments). 

I've seen weapon normal maps and meshes from a bunch of different high-end titles (courtesy of a modder friend) and haven't seen much trace of this being used. There are of course some popular cases where vertex normals are used for making meshes that have no normal maps like Alien Isolation, but I don't know of anything else that uses them frequently on any type of asset (weapons included). 

Hoping someone has more info on this. 

gnoop

  While normals editing was available for decades , even before normal maps had  became common,  CG people often have  no idea what they for.   Neither 3d soft developers.  "Why you might need it" was their common response usually.      I  personally did use them for faking  self shading of  some arkitekture details and trees only  before AO, normal maps  and projected shadows appeared. 

  Few years ago everyone was obsessed with an idea of some magic  baker/tangent space calculator  perfectly matching certain game engine  before it  gradually became clear it's basically irrelevant with just properly edited normals.     
 
In a word you are doing everything right   and 3d max is behind the current necessities as always.  Its normal editing tool set is amazingly outdated.  

FourtyNights

I agree that keeping custom vertex normals is somewhat fragile, even in Blender I use. I think breaking vertex normals with sharp edges (aka hard edges or smoothing groups), is much more practical, since a sharp edge is always an UV seam with normal mapped game res meshes, and those can be set/unset quite easily.

Custom vertex normals with beveled low poly meshes, which is usually the case when custom vertex normals are used... yeah, normal maps are having less gradients for compensating the low poly shading, AND they'll get mipmapped nicely...

...BUT is the positive impact on the quality THAT huge...? Well, maybe with hero hard surface assets that might give a nice quality boost... but then again, hero assets don't usually get too far away from the camera for mipmaps and LODs, so is it necessary then?

IMO, it's 50/50 whether to use custom vertex normals or not. So, at least I'm not worrying about the use of CVNs in the industry. I guess it also depends on the studio.

P.S. I don't have much experice with LODs and normal maps though. Just with LOD0s... which are the initial low poly models made by hand, lol.

Geosmith

It's definitely not a game changer, just an alternative. Either extra geometry, UV splits & hard edges or modifying vertex normals, one way or the other you'll have to make use of either the three to improve smoothing as you can't have the whole mesh one smoothing group most of the time.

I use plenty of hard edges over my meshes to split most smoothing up, mostly on very hard corners, but if the UVs allow it, I'll try keep it whole and mess with normals to make it better. I rarely chamfer edges to try and combine two sides of a hard corner, so in turn my use ends up being overall the chapest with the cleanest normal map, and like I said, the fastest for me because I've used them for a while. The worry is that others aren't familiar with them, which is the assumption I've gathered.

I guess it's somewhat silly to ask this question, like FourtyNights says it probably relies on the studio a lot than an answer someone can give.

poopipe

The main downside to using custom normals is that its a destructive solution to a problem and it isn't immediately obvious where and how it's been done on a mesh.
When the mesh ends up in someone else's hands you're relying on them to reproduce the same set of arbitrary information with literally nothing to work from. 

If you're projecting normals from other meshes or you have standardised scripts /rules etc. Then that problem goes away.

However . 

I am not a fan of widely using them in production for the simple reason that it's too easy for the information to get lost/broken. I'd rather models were built so they shade properly even if it is more expensive.

Noors

Whatever makes the thing looks good and doesn't require a shitload of time is good to take. Custom normals are widely used. I'm playing Mario odyssey : custom normals everywhere on environement assets. Custom doesn't necesseraly mean manully edited tho. As i stated elsewhere, the "industry" used broken normal map for years, rendered in wrong gamma space for years, because we all copy each others. Doesn't mean it's the right way to do.

Eric Chadwick

Can't do lowpoly foliage & grass without it. 

If with mechanical meshes that get edited a lot, use automated normal editing tools. 

TTools

+1 to @Eric Chadwick
Custom radialized normals are key to getting foliage and grass cards in-particular to light correctly.