Another problematic normal map thread...

kickzombies

Alright, so heres another noob to normal mapping with some problems.
I read through a lot of the "The ultimate be-all end-all normal mapping thread"
but couldn't really identify anything to help with my problem, its probably something really simple, me not understanding the concept fully? usually is.

First off I'm using Maya and its transfer maps options.
So these protruding objects are always getting skewed downwards and center wards and in no way is there position in the highpoly object doing this.
I played around with soft edges, surface normals and tangents, the cage, uv's. I took the high, low and cage and without cage, into xNormal and got slightly improved results but still a similar action and with added messed edges(my fault i know), nothing seems to help.

If anyone knows whats going on here any help would be greatly appreciated.

(The very bottom is my result from xNormal)

CheeseOnToast

Set all the lowpoly object's normals to be hard and bake again. Not as a final, but as a test. For something like this. you want the surface normal to be as flat and even as possible. If this fixes it, you'll need to add some more bevels/control edges to the lowpoly so the normal map doesn't have to work so hard. The offending faces should be as close to "normalmap blue" as possible.

Sorry for the crappy explanation, I'll try to explain myself better later if someone doesn't beat me to it. Do the test though, and see if that's the problem.

EarthQuake

Setting everything to hard edges will not actually help in maya, unless you set Match using to "Surface normals" which will give you really bad results as the projection mesh is no longer welded or "averaged". The problem here is that you simply do not have enough supporting geometry, so when your vertex normals are averaged around hard angles they get "bent". A good way to solve this is to add some more supporting edge loops, or bevel the corner edges, or both!

You can also look into rendering 2 maps, one using Geometry Normals(averaged cage) and one using Surface normals, and then combine these results in photoshop. I would only do this if you really really need it to be as low poly as possible. Otherwise supporting geometry will be good not only for an accurate bake, but also a better NM with less potential for smoothing errors.


red = bevel edges
blue = maybe bevel(test with just red first)
green = no need to bevel, as there are no details here that are specific enough to be noticed when skewed
pink = hard edge

play with this stuff here and you should be able to get a better understand of why you're getting the skewing.

[edit] Also a note, anywhere you plan to use hard edges you also need to have UV split, or else you will get artifacts on the shared "hard edge" in the final texture.

kickzombies

Oh! I didn't think that the bevels would have that much effect, before i was just playing with hard edges.
Still a bit of skewing but I have a start to understanding how to tweak that
much better now and a overall much better understanding of the process.

EQ that was tremendously helpful!

thank you both!

EarthQuake

Yeah even with bevels you'll still get some smoothing. Using a triple edge there(and edge on either side of your orig. edges) like you would to keep shape on a sub-d object would be the most accurate way to go, as that center loop then is very close to flat and just the 2 loops at the corners "bend".

I've had some good solutions for things like this by just cutting in a few more verts as well. You just get those extra 1 or 2 vertexes in there and it can help a lot. This can be a better solution than extra edge loops as that would essential double what you have now, and triple or more what you had to start.



Those circular extrusions and indents in the HP tend to be the most problematic thing when it comes to this stuff. It may also be worth noting that the farther your indents/extrusions stick out in the HP, the worse the skewing can be in the bake.


[edit] One final thing to consider, when you do very thin edge bevels you can run into resolution problems in your normals. Basically if you dont have enough pixels covering those areas you may get smoothing errors as there isnt enough space to represent the smoothing gradient in the texture. So its worth considering making those sorts of bevel extra big, even if its not entirely consistent with the HP shape.

eld

do like this:


Render one normalmap with the cage, render another one with straight rays to avoid the warping.


put them above eachother and start masking away the warped details to reveal the straight ray baked normalmap.


That way you'll get the nice beveled cagebaked corners with the correct front baked mid polygon details.


2. smoothingroups and split uv's on this one.

kickzombies

Problem solved, thanks guys.