Normal Map problems
polycounter lvl 18
I'm using max 7, following the tutorial Mop posted a while back. I've haven't had any problems untill now. Here is a screen of my problem:
As you can see, the lighting is all borked. Here is the normal map, where you can see the problem clearly.
Here is a screen of my two objects, showing the side of the high poly object that produces the problem in the normal map.
Now in the screen, the lighting error doesn't appear, it only happens when I project the normal map.
The lighting problem doesn't appear on the high poly mesh, it only shows up on the normal map when I project it.
Any ideas as to whats happening?
As you can see, the lighting is all borked. Here is the normal map, where you can see the problem clearly.
Here is a screen of my two objects, showing the side of the high poly object that produces the problem in the normal map.
Now in the screen, the lighting error doesn't appear, it only happens when I project the normal map.
The lighting problem doesn't appear on the high poly mesh, it only shows up on the normal map when I project it.
Any ideas as to whats happening?
Replies
Have you tried triangulating the low-poly mesh, or converting to Editable Mesh before rendering the normal map?
I don't know why this is happening, and I'd also like to know what's going on...
MoP
Here's what I did:
Reset both the high and low's XForm then converted both to Editable Poly again. Then I applied a meshsmooth to the high poly object (it didn't have any Sub-D's), but I popped it off the stack before I redid the projection for the normal map. So it was either the XForm reset, or the collasping to an Editable Poly.
I plan on doing more normal map work and will be back if I can repeat it and find a fix....
This is what it's generating at the moment, after resetting XForms on both low and high mesh... it's doing the same thing exactly 180 degrees round the wheel, on the other side.
Weird...
So far I have determined it's nothing to do with:
1) Resetting XForms.
2) Pivot points of both objects having to be identical.
3) The Lowpoly object being Editable Mesh or Editable Poly.
4) The orientation of the objects in world space, with or without reset XForms.
5) Whether the smoothing is native EPoly NURMS, Turbosmooth, or Meshsmooth in the stack.
6) Exporting and importing either or both objects from 3DS format (like an "extreme" XForm reset!).
7) The renderer. Scanline or mental ray makes no difference, nor does changing any of the renderer settings.
The testing continues... I'm beginning to think it's something to do with UV-maps, now. I re-mapped the wheel completely, and while the error still shows up in a similar capacity, it's on a different part of the normal-map! How odd!
I'm also noticing as I do more and more quick 512x512 normal-map renders, Max is getting slower and slower... the viewport runs fine but as soon as I change anything in the material editor, load or save a new image, it grinds to a halt for up to 20 seconds... ugh!
MoP
I've noticed max 7 slowing down too, but I thought it was just my machine, maybe not.... especially loading a new map in.
Have you ran into the problem where your mesh's edges will turn red and your normal map doesn't show up in the viewport? I've ran into that a couple of times, usually when adding and deleting lights from the scene.
And what's up with one side of the mesh being very very dark? It seems like when you 'render to texture' all the backfaces come out dark, and won't even be lit even when you put a light on them.
I'm going to do more testing tomorrow, see if I can find anything myself. I wish I could remember better what I did to fix my door. I know I didn't re-UV it, so I'm not sure if that will fix it either. Maybe I just lucked out. I do remember that I reset all of my materials and started over, so maybe that matters too. Could be several things....
[ QUOTE ]
6) Exporting and importing either or both objects from 3DS format (like an "extreme" XForm reset!).
[/ QUOTE ]
HA! that's funny!
And maX 7 is really slow for me sometimes aswell - I notice a lag when undoing operations - nothing huge but something that is definately annoying. Hopefully this sort of stuff will be sorted out for MAX 7.1 ?
You can use any Omni, Spot or Direct light, but as soon as you add a skylight the mesh turns to red wireframe. Just get rid of the Skylight and everything will be fine. I just tested that now.
MoP
The tesselation actually also makes the initial projection cage more accurate, although it's still better to Reset and just Push it back into shape.
The normal map it generated also has slight irregularities where the tesselated geometry was, instead of being perfectly smooth.
OK, I just re-uv-mapped the whole thing, and actually changed where the seams and layout of the map is going.
The error is still there, but in a different place. I also notice the error seems to be mirrored in each instance, but on a different axis.
That's the viewport display with this normal-map:
Compared with the original normal-map I was generating on these UV-maps:
It seems that the normal-map renderer and viewport shader deal very well with UV-seams, though... that new map I generated has a big seam all the way round the tyre, and the map is different pixel density on each side of the seam, yet it hides it very well in the viewport display. Shame about the glaring huge error, eh?
I bet you'll get an answer if you post on the Discreet webboard.
I just rendered out a World-Space normal-map now, and it is perfect, not a glitch in sight. But as soon as I switch back to tangent-space, the error appears again.
I've been looking through the Discreet forums, seems other people have this problem too, their solution so far appears to be "read the troubleshooting" and flip UV's around...
MoP
It's a smoothing group issue. I've always read/seen that you want your low mesh to be only 1 smoothing group. This is correct, but only after you render to texture your normal map in max 7. I sort of stumbled onto this too. I was working on a new model, and rendered the normals as a test. The image looked fine but I had some tweaking to do to the low poly model, which I did. And before I re-rendered the normals, I set all the smoothing groups to one group, like you're suppose to. And BOOM, problem. I go back, do an auto-smooth and FIXED, no problem.
I must be something with the way max 7 does its new render to texture. It seems like it uses the viewport for the projection, at least for the lighting information. So when you have only one smoothing group, it uses that lighting when rendering, for whatever reason. And I think we all know how crappy the lighting with one smoothing group can be too. I think this is also related to why one side of my model can be lit by a light in the viewport while the other can't. Its fine if I actually render it out.
And yes, my box for my door was one smoothing group, and I fixed it by auto-smoothing it, which I didn't remember till now.
Can anyone confirm??
Right, here's what I get with Smoothing Group 1 on the left-hand side of the wheel, and SG2 on the other side:
On the left is the smoothed, un-mapped low-poly mesh. You can see the seam produced by the 2 SG's.
On the right is the low-poly mesh with normal-map applied. When SG1 and SG2 are set as shown on the left, the normal-map looks fine, just with the seam being slightly obvious down the centre of the tyre.
HOWEVER! As soon as SG1 is applied to the whole model, the bug becomes apparent again! So the normal-maps still have a slight problem, but the smoothing group setup makes it less obvious... all i can say is "WTF!".
... Big rotating hairy arses floating in space. Cockmonsters. Other vague and menacing insults thrown in Max's general direction.
The smoothing group thing is definitely the issue, but changing the groups is like jumping out of the frying pan, into the fire, in my case. On an object like e_x's door, it's OK, because the edges are at 90 degrees so you don't notice the extra "fake seams" that no smoothing groups creates when rendering the normal-map. However in the case of my smooth tyre, it's very obvious. I've tried every combination of smoothing groups I can think of, and the error is always there, mocking me. It just comes and goes to varying degrees depending on how I set up the smoothing. But it's ALWAYS there. Someone should probably hit me before I go insane here.
MoP
Thanks for you help Mop
Your normal map is attempting to complete a job, which is to display how the model will appear when its lighted. Your smoothing groups are also attempting to do the same job as the normal map. One smoothing group can work just fine with a normal map...adding more than one causes confusion in the work force...and you get errors in rendering like that.
So...each Element (an element by max's definition of Element), should only have 1 smoothing group assigned to it. Hope this helps...
Edit: If it helps, think of your normal map as your smoothing groups...those other "smoothing groups"...have been replaced...given the pink slip...lol told never to clock in again...so don't have more than one on your low poly model at anytime.
Sorry! Nice try though
Ah, perhaps I didn't explain properly. My first test (see my very first wheel image in this thread) was generated from a low-poly mesh with only 1 smoothing group.
Sorry! Nice try though
[/ QUOTE ]
Ahhh...perhaps if you or E-X could send the .max file to me @ IllusionsCG@gmail.com, I could take a look. A picture, or in this case a file, is worth a thousand words...
So...I still have no idea how to fix this...but perhaps this will help a bit.
Edit: E_X have not recieved your door yet for viewing. I'll tell you tomorrow if I've recieved it, and if I haven't just resend it and I'll take a look
Messing around with the UV-maps only seems to make the error move around, not get rid of the error at all... which is annoying.
Smoothing group changes get rid of the error, but add other problems - bah!
MoP
Edit: Alrighty, I don't know whats causing it but I know where its originating from...the low poly mesh. Somehow...theres something wrong with it. In your case lucky enough its just a cube, and is pretty easy to rebuild. I rebuilt your low poly door, re-uvmapped it, and...it rendered fine without the error. Odd...
there has to be a better solution to this though, cause going in and correcting this by either rebuilding the low poly model, or paint-correcting the normal map in photoshop would be a real bitch on some more complex models...
Thanks for looking at it.
This low-poly mesh has less segments than the other one, and is constructed in a fairly different way, geometry-wise.
The exact same error is appearing in the exact same place.
It's not down to geometry then...
Ohhh! This is an interesting find... I accidentally hit Render to Texture when the hi-res mesh was Hidden in the scene - naturally it rendered it as red to show the rays hit nothing - yet the error still appears in the same place on the low-poly model being displayed in the viewport! What the hell!
And whatever I do to the UV-maps, the error is always there, just in a different place depending on how I place the seams and rotate or mirror the UV's ... grr!
The quest goes on...
Edit: MoP, its definately the Low Poly mesh or however you're modelling it. I just redid your wheel the same way I did E_X's door and it rendered out a normal map perfectly. All I did to make the wheel was use a standard cylinder, then used a projection mapping, opened up UVW, and ran flatten mapping on it...and it came out fine.
The problem is... 3D Studio Max 7.
The evidence:
On the left, normal-map rendered and displayed on the low-polygon mesh in NVidia's Melody tool. The meshes were exported from Max in .obj format.
On the right, the exact same normal-map from Melody, displayed on the exact same low-poly model, in Max's viewport with the DirectX shader enabled. Notice the errors and general dodgy shading.
So, it looks like the problem is with Max in general, not any specific method. The errors show up on the flat maps when rendered from Max's normal-mapper, but do not appear on the maps rendered from Melody. Bah!
MoP
Either way , we all have troubles displaying and using things that should just work ..
When I was testing the model shown in my last post, I made a new material, turned on "DX Display of Standard Material", added a Normal Bump map in the Bump slot, and AS SOON AS I DID THAT, the shading error appeared. No bitmap loaded. Just the natural shading with the pixel shader enabled, has that "dent" of a shading bug, as you can see on the right-hand-side of the above image.
What the hell!
MoP
It's not a max problem and they wont fix it, it's just the way it works, in any app i'm afraid.
Usuall workarounds are to either use proper smoothing groups (too high= bad results, if they look bad in a OGl viewport they will bake bad)), unweld pooints for baking were needed, or subdivide the lowpoly mesh for baking, that will reduce the size of the areas being baked wrong.
And smooth groups certainly don't have anything to do with the "dent" that's appearing as soon as I add a "Normal Bump" material to the object in Max, even without loading a file - that's GOT to be a Max bug, or at least a DirectX bug.
If removing the smoothing groups fixed the problem in your model then it should be the same problem somehow. Did you try subdividing the lowpoly mesh just for baking the map ?
That affects the smoothing groups to some degree, might be enough of a change to fix the problem.
However, oddly enough, you are right on the smoothing groups point. I tried changing the SG's (they pretty much match the UV-splits now, I wonder if that helps?) and then baked the map. It came out fine! Hooray!
However this still does not explain why Melody can render a perfect normal-map from a low-poly object with all 1 smoothing group, yet Max cannot.
Also Max's lighting is still weirded in this fashion - I make an omni light, move it to the right of the object, and everything lights fine.
Move the light to the other side of the object, most of it remains dark, except some bits which light up to some extent. It doesn't really make sense, because if the normals were flipped, surely they would light up when the light is on the "wrong" side of them? In this case they just don't light up at all from any angle...
Hmm!
Maybe try the RESET XFORM - its always handy in situations like these!
There's quite a few ways to handle normalmaps, some dont support mirrored UV's, this and that, maybe Max doesn't has the smartest implementation.
Also Max's lighting is still weirded in this fashion - I make an omni light, move it to the right of the object, and everything lights fine.
Move the light to the other side of the object, most of it remains dark, except some bits which light up to some extent.
[/ QUOTE ]
Did you ever figure this out? I was playing around with Max's normal mapping and I think it has to do with the Tangent, Local XYZ, Screen, and World options in the Normal Bump Options and in the Projection Options. Make sure they match. I had the same problem when I used Local XYZ in the Projection Options, but didn't change the default Tangent in the Normal Bump Options. Changing to Local XYZ in the Normal Bump solved the problem.
Eric: There's been a few complaints of this type on the Discreet forums, so I'm assuming it has been logged - where can I go to check?
If you render it in any method but tangent, it comes out fine, no errors. If you want to render in tangent with no errors, you have to apply smoothing groups, at least one per UV element. Then you will get no errors. The problem with this is that you now have butt fugly seams in your model, where they don't line up. So the only answer I can come up with atm, is to render the light bake in max, but do the normals on the exact same model in Melody. Melody knows how to handle xyz, as does every other single frickin normal mapping program. I don't know how this got passed Discreets testing, but I am assuming they did not have anyone who knows how normal maps are made in a production pipeline test it. So till discreet comes in line with proper workflow, everyone use melody. 8-)
Defect Report Submission Form
(Found by going here and choosing Feedback from the left panel.)
never mind, just tried, no worky, tried several other things also, no work,
I'm gonna see what happens on the current high-poly model I'm doing. Almost finished the low-poly mesh, gonna UV-map it then do normal-map renders from 3dsmax, and hope it works.
If not, Melody is the best bet...