Max displays normal maps incorrectly, when using nitrous direct3d.

deLuxe

First post. Let me use this oppertunity to say that this community is awesome and I appreciate the honest feedback people get.

So, ever since a hard drive reformat and subsequent re-installing of everything I have the issue that Max 2016 displays my normal incorrectly - but only when using hardware shading. When I'm using nitrous software, all is good. I'm 100% sure that the maps or models are not in any way broken - in fact, on my laptop they work fine, as well as in the target engine.


Above we see the issue: This is just a simple cylinder with some fancyness baked on it. It is almost as if the hardware accelerated shading disregards the gamma/LUT settings.

Here's the info that might be useful:

• The maps are baked within Max using Scanline (bonus Q: has anyone gotten mental ray to produce acceptable normal maps?).
• Display them in the viewport using a standard material -> bump -> normal bump -> bitmap.
• Gamma/LUT has been turned off, alternatively, I've tried manual gamma = 1.0 override.
• Smoothing Groups and UV shells have been set correctly.

So just for fun, I decided to use a 'blank' normal map (filled with RGB = 128, 128, 255). Still, I got incorrect shading - reverting to correct shading when turning off the bump in the mat editor.

So... Anyone know the issue here, or perhaps have a workaround that permits me to view normals withing the viewport? I'm using somewhat of an ancient workflow switching between max and photoshop, and the ability to see how my normals are behaving is hugely beneficial.

Eric Chadwick

Short answer: don't use the Max viewport to validate normal maps. 

For flawless results the baker and the renderer have to be synched to the same tangent space. Max's baker is only synched to its scanline renderer. 

If you want to see your model correctly, choose your game engine, find out what tangent space it uses and bake to that. Marmoset Toolbag is a great choice for baking these days.

deLuxe

Short reply:

I know using viewports to validate maps is considered sketchy practice, but up to quite recently, I've been able to get decent results with it - at least decent enough to know that it would somewhat work. The issue now is that all of a sudden max has decided to just completely mess up display of normals (in the situation described above). Thats the weird thing.

I'll have a look at marmoset one of these days though . For now I'm actually somewhat keen te keep the number of programs down.

Eric Chadwick

You could try using Shader FX to write your own viewport shader. Not sure what tangent space it expects though.

spacefrog

I just wonder why you use Nitrous DX9 mode ? Which Max version are you using ? AFAIK Nitrous DX9 mode is not developed or supported any longer and you should keep using recommended Nitrous DX11 mode.
In my test using 3ds Max 2018 your normal map ( cropped from your jpg ) shades quite fine in Nitrous DX11  viewport mode. After testing and switching back to Nitrous DX9 mode nothing normal map related really works in the viewport any longer

deLuxe

That'll be it then. Being a poor boy my GPU doesn't support DX11. Laptop has max 2015, desktop has 2016. I'm guessing that'll be it then. Thanks for the pointer there!

Edit. Working in max 2015 solves the issue. Thnx for the effort