3dsmax favors local-space normal maps?
polycounter lvl 19
After doing some tests the past few days, it seems that 3dsmax prefers to properly display local space normal maps over tangent space normal maps.
These two objects here have the exact same DirectX FX shader and grey diffuse texture. Notice how the local-space looks how it should.
Has anyone else here experienced the same issues? And if so, is there any way around it?
These two objects here have the exact same DirectX FX shader and grey diffuse texture. Notice how the local-space looks how it should.
Has anyone else here experienced the same issues? And if so, is there any way around it?
Replies
It's just max being assy... if you generated/rendered in Doom3, it'd be fine. Just for some reason Max's tangent-space display likes to kinda overlay the lowpoly smoothing onto the normal-map smoothing, so it looks crap.
The tangents and bitangents need to be calculated per-vertex from the UV coordinates of the model. Once they're created, they can be stored in an unused UV channel. Then the light vector can be transformed from world space into tangent space, and the surface will be lit correctly (most of the time... there are exceptions like the poles of a sphere, so I'm told).
Here's some code if someone wants to whip up a proper Max plugin or script...
http://www.terathon.com/code/tangent.html
...though you'd also have to have a viewport shader that'd be able to use the resulting vectors.
Hopefully someone here is crafty with code/maxscript, and is willing to take this up.
I'm clueless on this code stuff, though. If someone could make it support 3 lights, transparency, and specular, I'll have your baby
And using his scripts, I still get f'd up shading like the above pic.
On a related note - did you guys notice that they fixed the issue with Normal map seams at UV borders in Max 8? Yeah! Also, you can now mirror your UVs and the lighting is correct. Some nice normal mapping improvements in this version.
Am I right in thinking that Max's RTT renders the normal-maps perfectly correctly, just the display is screwy? Or is it linked in some way?
Yeah, the Max8 fixes are good, seams and mirrored UV's are now handled very well, however those artifacts on objects with 90-degree angles (like Vassago's example here) still display nastily using the standard shaders.
Great work, man, I really appreciate it, and I'm sure hundreds of other people do too!
MoP - I think that Max does in fact render the maps correctly. I had a freind look at tangent normal map above in his engine, and it looked perfect. I figured that Discreet would have resolved this issue by now, but I guess no one has spoken to them about it :P
Can't wait for those scripts, Ben!
http://www.monitorstudios.com/bcloward/test.fx
As you can see, the Max default shader doesn't even load a texture larger than 512x512, resulting in horrible blurring. Also, the normals along sharp angle changes are screwed with the vertex shading.
Your older shader works a whole lot better, but there are a few areas where it's noticeable that the vertex shading is darkening areas inappropriately.
The new test shader seems to work perfectly! Nice one!
It does seem to be an improvement, though, as shown on my screenshot before, a few obvious vertex shading artifacts were removed completely, and nothing got noticeably worse, only better.
Also, if you keep developing this, is there any chance to have a checkbox or drop-down menu to choose which way the X- and Y-axis shading is being calculated? Your old shader and your new shader require the Y-axis (green channel) to be flipped between them. An option to choose that would be hugely beneficial (since there are also game engines which use different methods, and not having to manually invert a channel in Photoshop would save time).
Thanks again, great work! Hope you get something resolved...
Adding a check box to choose which way the channels should go is very easy. Yes, I will add that to all my shaders. It's a handy feature - especially for a shader that's being used by several different companies.
Solving the sharp angle & long edge issue is going to be a lot harder. The reason that the local space normal map looks so much nicer is that that type of map doesn't use the vertex normals from the low poly model at all. It only uses the data in the map. The problem with a tangent space normal map is that the data in the map is recorded as an offset from the low poly normal instead of a normal that can stand on its own. Generally just using an offset is ok, but in extreme cases like sharp angles, the vertex normals of the low poly model are going out diagonally from the surface and don't match what the actual surface normals should be at all. That means that the tangant space normal map is really fighting against the screwy normals of the model to bring them back into an orientation that's more reasonable. Somewhere along the line - either when the normal map was generated, or when the lighting is calculated, something is not being done quite right. I don't know what it is.
I know that in the case of this particular model, you can get better results by making seperate smoothing groups for the polygons on the back side of the fountain and also the bottom. That might help. You could also add one more edge loop around the middle that cuts the shading problems in half. This would give you some vertex normals that were a little less extreme. Re-generate the normal map once you've made these changes and see if that helps.
You might also try generating your normal map in ATi's NormalMapper or with ORB to see if you get any better results from those.
The real strength of a tangant space map is that it is deformable - so you can use it on a skinned character. You can't do that with a local space map. In the case of this fountain, maybe you're just better off using a local space map since it will never deform? :0)
I don't see why tangent space normal maps work that way, then. The whole purpose of a normal map is a have the lighting of the high res mesh. How can this happen when it's calulating the low res smoothing on top of it? I thought that it should be behaving just like the localspace map is, in max. grr...
As far as the smoothing goes, that's not a solution. It's going to break the smooth edges of the mesh corners, which completely negates the benefits of the normal map. I don't want hard creases on the edges of the mesh...
I'll have to try hacking the mesh up and re-baking the normals. If I do in fact have to use local-space normal maps, do the same principles apply when overlaying detail in photoshop? I mean, they're completely differant in color, so I don't know if it's the same process or not.
Any help would be great..
**edit
I forgot to say, that I sent this in to a freind, and they tested it in their engine. This is the same tangent space normal map that I used in the above viewport shader tests:
The problem still seems directly linked to Max's inability to properly display tangent-space normal maps in the viewport.
How can this happen when it's calulating the low res smoothing on top of it? I thought that it should be behaving just like the localspace map is, in max.
[/ QUOTE ]
The different methods work differently.
World-space and local-space (or object-space, same thing) use the same style of normal map, normals are encoded with world-space values.
World-space: The shader takes the vector of the incoming light and compares that with the per-pixel normals from the map, then lights the surface. Mesh cannot be rotated differently than the original normal-map-baking orientation, or the lighting breaks down. But world- is the cheapest method, shader-instruction wise.
Local-space: Same as world, except the mesh object can be rotated away from the original bake orientation. This rotation is simply added into the shader's duties, transforming the light vector into the local orientation of the mesh. A little more processing that world- but not by much.
Tangent-space uses a different style, as Ben said the normals are encoded in reference to the vertex normals of the mesh. So now the mesh can not only be rotated but also deformed (skinned characters). The shader has to do a bit more work with tangent maps than either world- or local-, because it has to transform the light vectors from world space into the tangent space for each vertex, and henceforth onto each texel (pixel on the mesh).
[ QUOTE ]
If I do in fact have to use local-space normal maps, do the same principles apply when overlaying detail in photoshop?
[/ QUOTE ]The "standard" overlay method works for tangent-space maps, but not really for world- or local- maps (world- and local- use same map). World-space maps need an overlay that's actually colored in world-space itself, not an easy task. There are ways however, like using a special lighting setup to bake a grayscale bump map into world-space, which you could then overlay (or somehow) add to your existing map. Never tried this myself.
[ QUOTE ]
I sent this in to a freind, and they tested it in their engine. This is the same tangent space normal map that I used in the above viewport shader tests:
[/ QUOTE ]Looks good! The game exporter is generating the proper tangents and bitangents, so the lighting rocks.
It took me a bit to learn all this, from Ben and others, so I might as well share. Good luck Vassago, I know it can be confusing. It certainly was for me.
Vassago: I'm wondering if the engine your friend tested that in, is an OpenGL or DirectX graphics engine? Do you know?
If it's DirectX, then what could it be that's causing Max's hardware renderer to screw up the display, if it's possible to have a perfectly working tangent-space normal map in a different hardware renderer using the same SDK?
Could it be that it's an inherent Max fault, and that no amount of HLSL shader coding will be able to get around it?
I hope that's not the case, but if it is, then... well, damn!
It's got to be a problem with 3dsmax itself. Like Eric said, the tangents and bit tangents needs to be recalulated upon export/viewing.
It's very unfortuneate though, since I now have no way to accurately look at my work. It's probably going to be quite some time still, until our new engine supports normal mapping.
It wouldn't be hard for someone to write a little utility that calculated those, then stored them in a modifier or somesuch, then write a shader that would take advantage of that data. Coders do it all the time for game engines, and Max's viewport is essentially a mini game engine.
But I'm not a coder...
It's probably a simple matter of priorities. They are on a cyclical dev cycle after all, and features must be prioritized.
Though frustration is certainly exaggerating my anger
With the whole normal mapping thing taking over, I just would have thought that by now, there would be a standard set of tools out there for people to use. Not to mention function properly.
Sure some tools have been late in coming, or not at all, but that's true of every 3d package out there. When you're on the cutting edge, it hurts some times.
- Side-by-side comparisons of what your model looks like using lots of different normal map display methods - as many as you can think of - kinda like the image that MoP made above. You might also include what the model looks like when you render it. Be sure to mark the spots on the image where the lighting problems are so it's very obvious.
- Include your max file with all of the maps required so that they can reproduce the bug.
- In the material editor of the file set up several materials, each with a different method of displaying the normal map and include instructions on how to recreate the problem and view the model with each different material/shader.
- Be as clear as you can and especially emphasize the fact that the model looks perfect in an external game engine and also in Max's scanline renderer.
- Include a link in your bug submission to this forum thread so they can come and get more details if they need them.
I'd really like to hear a reply from Autodesk on this one! If they do fix it (they really need to) then you can feel all cool since you helped make the software better!
I've gotten onto their tech support forum, but they have an attachment size limit of 200kb.....
I've had no response before. I've just accepted that the message was probably passed on to the suitable folks, or stuck into the queue.
If attachment size is an issue, why not host the files yourself? If you don't have a site, free upload services abound.
But it is all worth it, IMO. Good software depends on user feedback.
We're aware of the issue but still seeking the file in question. We need both the low and high-res objects in .max files, and we also need the generated normal map to verify the problem. Information about your graphics card and driver can be helpful too.
For general suport issues, please use http://support.discreet.com/webboard/wbpx.dll/%7E3dsmax
To log individual bugs, don't be shy, please use http://usa.autodesk.com/adsk/servlet/item?siteID=123112&id=5600504&linkID=5573345
I'd like reiterate everything Ben Cloward said in his previous post about bug logging. We do take this seriously and I cannot overstate the importance of logging a clear, methodical, comprehensive bug report. Although an issue may seem of paramount importance to an individual user, we do a lot of prioritizing since we guggle several market segments.
Regarding the tangents and binormals; yes of course Max calculates these for both rendering and viewport display. If the viewport used some kind of technique that did not rely on tangents/binormals, it would be mathematically nonsense and would look much much worse to the user. The tangents/binormals are not stored in a mapping channel, and after all they would change every frame if the object is deforming. So they're stored only in the vertex buffer. The problem with tangents is that there are many ways to calculate them, and problems often arise when maps are encoded and decoded with techniques that don't match. In Max 7, the RTTNormalMap.fx probably had an issue here, but I'm thinking it's fixed in Max8. Which version are you using?
Also, I fully agree that you should always use Local Space maps for objects that are not deforming. However, touching these up in Photoshop can be almost impossible, so you might stick with Tangent if Photoshop'ing is a must.
Cheers,
Michaelson Britt
Software Developer
Autodesk Media & Entertainment
I'm using the Max8 Demo, for all of the above tests. This includes modeling, baking, viewing and rendering.
About the tangents - since everyone here is having the same issues, would it be safe to say that Max is handling them wrong? I know there are multiple ways to calculate them, but if they show up correct in nearly every game engine, why couldn't they show up correct in the viewport?
Thanks again for the reply, and for posting the information here for the community.
The lighting will only matter if you're rendering a complete map including diffuse and shadows and stuff... just the RGB normal map is not affected by any lighting.
I'd use a Skylight with advanced lighting turned on to get good ambient lighting from a high-poly object if I was baking all the textures. However, if you're not rendering out lighting, don't use the Skylight because it will slow down the render-to-texture process needlessly.
That maybe made sense...?
Does RTT store the tangent/bitangent data when it creates a normal map? I can't find any options to turn it on, at least in Max 7. It seems to me both the scanline dlm and viewport fx shaders would need to access that data for it to be useful.
I think it would also be helpful if the RTT data was easily exportable via an SDK pointer, so it could go out with the rest of the game data. Might help to make it a part of the helpful Game Export Interface example code.
Thanks again.
I wonder if it's related to this topic. Look at this. Same model, 1 smoothing group both models, Max Render to texture Vs Melody. May I be doing something wrong with max normalmapper? I don't see any button that says "create beautiful normal map"
Thanks all.
http://membres.lycos.fr/dreamdesigner/coses/maxmelody.JPG
(Mop, it was 3dsmax viewport
But since those were both displayed in Max, it doesn't matter, and I guess you've fixed it now anyway