I generated a normal map in Max 8 the other day from two models, a high and a low poly, both with a single smoothing group. For the large majority of the normal map, everything comes through, but it seems to contain vertex lighting between certain points.
I don't really know what's causing this, because the vertex lighting is apparant in the low poly version, but not in the smoothed high poly. I read a few tutorials, some old posts on Polycount, but can't really find anything with the same problem.
On the other hand, my normal maps have combined the some of the most primitive lighting techniques with the most advanced lighting techniques, so I guess I should be proud.
Edit: It's also worth noting the problem only occurs in the Left-Right (Red) and Up-Down (Green) channels.
Replies
If you think about how a normal map works, it contains information about the highpoly normal relative to the lowpoly normal (hence "local space"), so it's actually quite natural to see triangles showing up in the shading, especially of flat surfaces rendered down to a single smoothing group.
It looks like you might wanna turn some edges in your lowpoly to get a nicer output though.
I'd imagine if you preview this normalmap applied to the lowpoly it probably looks ok.
Also, thanks for the quick comments. I'll play around with the tris, look into Chris' suggestions, and keep in mind how normal maps & smoothing groups will behave very diferently depending on the engine.
The normal map preview looks extremely messed up. I took a screen of a viewport shot vs. a render. The normal map and UV layout hasn't been tweaked yet, so there are some raymisses, overreaching UVs, etc.
Also, thanks for the quick comments. I'll play around with the tris, look into Chris' suggestions, and keep in mind how normal maps & smoothing groups will behave very diferently depending on the engine.
[/ QUOTE ]
Yep just like i thought, now depending on what you're actually doing for this you have a few options:
If you're just using this as a showpeice and never plan to put it in a game engine dont worry about it, just render it in max. Also if you plan to use this in UE3 it *should* be fine and display correctly.
If you want to put this in an actual game engine, or even just view it correctly in the viewport on in a program like xnormal you have a few options:
1. Set up smoothing groups on your low poly model, and re-render your normals map so that you dont have such harsh angles on the normals. This will look like ass because you'll have a bunch of hard edges everywhere.
2. Add tons and tons of chamfers to your mesh so it will actually work with a tangent space map, usually this will make your polycount about 3x as high.
3. Use object space normals maps, this is what i've been doing on the project that i'm on. Your engine has to support it but its really the best option(outside of figuring out what magic UE3 and Max's renderer use) but also has some limitations.
Additionally if you want to throw this in d3 thats again a total different story, and is more dependant on your uvs than anything else....... Really, its a big fucking headache no matter what you do, enjoy
Learned shitloads from every step of creation of this thing. From High Poly hard surface modeling to a 3ds Max workflow vs. Maya. I hope I'll have time to finish it, tomorrows my first day of work and I'm wondering if I'll take it in stride.