How engines shade prominent normal map details on shaded side of geometry ?

gnoop

Is there a way to kill illuminated details on shaded side of geometry  with shadow maps off?

Obscura

I think its only possible with shadow maps off, by disabling the normal map on the shaded side. You can find the dot product between the light vector and vertex normals, clamp the result so its 0-1 and blendbetween flat normals and your normal map. This won't  look good on anything more complex than a box though. The normal map will loose intensity going towards shaded areas. You could use a more complex formula to push the disabled area to be only inside the fully shaded pixels, but then the transition would still have illuminated pixels. 

So most likely no, not really possible. Maybe an ambient occlusion map or ssao could help reducing the effect.

gnoop

Thanks Obscura

 We tried something like you described and it does cause issues. Hoped there may be  some simple way 

Farfarer

You could take the smaller of the dot products for light/tangent-normal and light/vertex-normal, but it won't be perfect.

gnoop

Thanks Farfarer     I guess it will reveal every hard edge and would make  rounding them through normal map  impossible.

Eric Chadwick

How often is this really apparent though, in real use cases, not with a made-up smoketest?

pior

I would say ... very apparent on human faces baked from highres, and shown in an environment not allowing for control on incident lighting direction in relation to the camera. Hence the Mass Effect look : glowing lower eyelids and nostrils (glowing as in, catching highlights when they shouldn't.)



Good tricks to prevent it would involve carefully editing vertex normals ... maybe a little bit of manual smoothing in the normalmap ... and some careful texture authoring to prevent such areas to catch to much light info. Maybe also going as far as having a custom shader lowering the incident light angle on faces overall. I do wonder if Black Desert is doing any of that actually, since their character faces are remarkably smooth in that regard yet they are still using tech and assets from nearly a decade ago, and with much less control on the camera and lighting than in a single player RPG cutscene.

Eric Chadwick

Shouldn't baked ambient occlusion be taking care of those areas?

Mass Effect is not using all direct lighting, must be IBL or spherical harmonics, which is "ambient" light and so it should be occluded in crevices? 

pior

That's the thing - AO takes care of shading the cracks, but it doesn't occlude convex edges like the thin details of nostrils, or the up-facing edge of lower eyelids.

This is basically the reason why so many "high end" games can look so cheap as far as characters are concerned, regardless of how polished the source highres art is, or how powerful an engine is. Unfortunately since this seems like a minor issue during production (or even, a problem that goes completely unnoticed when reviewing fancy screenshots of sculpts) this is very often overlooked. MVCI has this all over its ingame character art, the source high of which is probably great.

rollin

It's not that easy. Bc you would also have to consider cases that are solved only by.. self shadowing.

You can do some tricks but overall you can't know anything about the big picture if you're just looking at a single pixel.

Best intermediate solution might be to run the character self shadowing process on it's own setup (as unity's blacksmith demo was doing) to get a higher shadow res as the rest of the world

Eric Chadwick

Yep, fault of the lighting, not normal mapping.

There's a set of slides from a conference a few years ago, which I just saw again yesterday, that attempts to deal with issues like this. I'll dig it up. 

Eric Chadwick

Ah, here it was.

 http://research.tri-ace.com/Data/cedec2011_RealtimePBR_Implementation_e.pptx

• ambient vs specular occlusion
• conflicts with point lights
• reflection corrections
• etc. 

RN

gnoop said:

Is there a way to kill illuminated details on shaded side of geometry with shadow maps off?

Obscura said:

You can find the dot product between the light vector and vertex normals, clamp the result so its 0-1 and blendbetween flat normals and your normal map. This won't  look good on anything more complex than a box though. The normal map will loose intensity going towards shaded areas.

I'm not sure if this is what you tried, instead of blending the normals, multiplying the light factors.

If I understand what @Obscura suggested, you have this:

A ) Blending normals:

<code>vec3 normalMapNormal = texture2D(normalMapTexture, uv).xyz;
vec3 blendedNormal = normalize(mix(vertexNormal, normalMapNormal, vertexLight));
vec4 color = saturate(dot(lightDirection, blendedNormal)) * diffuse;
(...)float vertexLight = saturate(dot(lightDirection, vertexNormal));<br>

B ) Multiplying light factors:

<code>float normalMapLight = saturate(dot(lightDirection, normalMapNormal));<br>float vertexLight = saturate(dot(lightDirection, vertexNormal));
vec4 color = vertexLight * normalMapLight * diffuse;
(...)
vec3 normalMapNormal = texture2D(normalMapTexture, uv).xyz;<br>

gnoop

Thanks for posting the example RN     Still I bet with hard edge / split normals  nothing would help really  because of too much difference in between vertex normals and pixel ones.   

I hoped for some magic trick but seems it's just not possible.  Would  work probably with all smooth + extra loops models.

As of nostrils and such details It the thing where simpler hi and low res geo sometimes look better . I also regularly hand paint normal maps to make details less contrast in those areas.    Still it's always a huge time hog  with typical non stop miss and hit process.

Obscura

dont hand paint normal maps it should be illegal.

gnoop

 I am guilty  

it's bad practice indeed but in some specific cases it's still only way

RN

gnoop said:

Still I bet with hard edge / split normals  nothing would help really because of too much difference in between vertex normals and pixel ones.  

I'm thinking the shader doesn't know what split normals is, when the mesh is being rendered it's already split at the edges.
The shader will get the flat normals (the ones that make the mesh look good), it's worth testing.

Eric Chadwick

Just paint nostrils and eyelid edges black/rough. Boom.

gnoop

Nostrils - yeah it would help, eyelids - would look weird probably. In many other cases it wouldn't help much at all.  I initially looked for a solution to fix  instanced white painted  tire stacks not using self shadowing. Very little space to paint something black