As you can see in the first image I'm getting some interesting results with my tests. Looks as though the lowpoly smoothing group was baked into the normal map? There's also some sort of noise as well, looks bumpy. And too add to this I'm having one hell of time trying to get the high poly edged to line up with the low in the normal map. I tried some other things and it seems I may need to give both the high and low a little more thickness in order to achieve better results. Suggestions?
As far as that edge goes, you're never going to get that to line up perfectly unless you add more polys around the edge of the lowpoly.
The process of normal mapping is exactly like trying to fit a round peg into a square hole - sometimes it just doesn't work, especially around chunky low poly edges.
You might want to try and add a turbosmooth on top of your projection modifier, sometimes (but not always) this gives cleaner normal maps.
I tried xNormal (thanks jogshy) and these were my results, then took that map and dropped it into Max. These were my results.
I'm a little frustrated (read pretty angry) that Max isn't cooperating. However, it seems xNormal is my solution to this problem for now. Thanks for the input folks.
Edit: Just tested the normal map in maya and it looks just as good as xNormal. What's up with Max??
I got some errors like this on the past. Model, Smooth groups, UVs, Normal maps all looking fine, but when viweing it all on Max got something like those bugs. On Xnormal or anyother viewer it looked fine too. So i tryed on second computer on Max and it was looking fine. Guess it might be just some Max crap. Didn't try reset Xform then and it might be a good idea.
Anyway ended up igonoring it (it was just on a small almost hided part so no big deal) and continue my model that way.
Resetting Xform does squat. Flipping the green channel made it render proper for the most part, but there's still all that weird noise when I view it using DirectX. Viewing the model+map through a render looks ok, but I can still see the smoothing groups and the map looks pretty flat. Again, thanks for the help. I think I'll stick with xNormal for this project.
The map is actually a 512x512 I was testing. I cropped out the excess black for posting purposes. In Max I'm rendering a TGA. I'll try flipping some channels again when I get the chance, perhaps I did it wrong initially.
The edges you see in the normal map are necessary, Max's shader has to compensate for the low-poly normals to give you smooth shading.
Max, like all other normalmap bakers, creates tangents & bitangents at bake time. It then uses these afterwards to convert the incoming light vector into the right orientations to match your tangentspace UVs, for consistent lighting direction. If you change your mesh between baking and shading, you could remove those tangents.
Replies
Your cage is messed up and clipping into areas it should not.
What app are you using? I can get specific if I knew.
The process of normal mapping is exactly like trying to fit a round peg into a square hole - sometimes it just doesn't work, especially around chunky low poly edges.
You might want to try and add a turbosmooth on top of your projection modifier, sometimes (but not always) this gives cleaner normal maps.
I'm a little frustrated (read pretty angry) that Max isn't cooperating. However, it seems xNormal is my solution to this problem for now. Thanks for the input folks.
Edit: Just tested the normal map in maya and it looks just as good as xNormal. What's up with Max??
Anyway ended up igonoring it (it was just on a small almost hided part so no big deal) and continue my model that way.
What file format are you rendering the normalmap to in Max? Could also be causing problems.
The edges you see in the normal map are necessary, Max's shader has to compensate for the low-poly normals to give you smooth shading.
Max, like all other normalmap bakers, creates tangents & bitangents at bake time. It then uses these afterwards to convert the incoming light vector into the right orientations to match your tangentspace UVs, for consistent lighting direction. If you change your mesh between baking and shading, you could remove those tangents.