weird artifact in normal map

gamenoob

hi,
look at the map.

isn't the marked area is weird? I baked it from xnormal. Any idea why is this happening?

Angry Beaver

Is your cage intersecting your model? it almost seems like the mode crosses the cage at those points completely flipping the angle it's projecting the normals at.

gamenoob

What is cage? How it is related to baking process? I am kinda beginner at baking stage.

Angry Beaver

What happens is when it it tries to bake a high poly mesh onto a low poly model it establishes a "cage" around the low poly model. This cage tells it from what direction it should project the high poly onto the low poly to make the normal map. Making this cage poorly can really bugger up your normal map baking, controlling and tweaking it is an advanced topic, usually the xNormal defaults would be fine so I'm not sure what's up. I was guessing that your high poly or even low poly mesh might of extended outside of the cage causing these issues.

gamenoob

I have done a thing. I ticked the option called "use cage". Then I did "right click>browse external cage file",then I select the same low poly mesh. Now the artifact is gone:\. I want to know more about cage. I want to control it sothat I may be able to extract the best out of my sweet high poly. Do you know some good documentation or tutorial etc about it(also more generally about baking process)? polycount wiki does not have one for caging.It would be a great help. My high poly model looks very good contrast to my ugly low poly model. I want to end this problem.

Angry Beaver

This is a good discussion on it I picked up just by searching this forum specifically for xNormals + Cage. http://www.polycount.com/forum/showthread.php?t=120232&highlight=xnormal+cage

I don't have anything good bookmarked so I'd have to go searching but I hope that helps.

Eric Chadwick

Sticky: Understanding averaged normals and ray projection/Who put waviness in my normal map?

gamenoob

Thanks guys.
Hmm,it seems that cage is nothing but a copy of my low poly mesh. While ray tracing high poly mesh resides between low poly and copy of low poly mesh aka cage. So,before I selected only one low poly mesh.Now I select this same mesh twice.First as low poly mesh,second as cage. I see that I can add more mesh in low poly section in xnormal. I wonder why? However,the weird artifact is gone but another just came in.

I was trying to make a door.
This was my base mesh.



Then I sculpted it and this is high poly mesh.




And low poly mesh in game engine.




Normal mapped low poly mesh,




Continued...

gamenoob

Now look at the problem.
Here,







Some details got themselves displaced and place it on unusual position.
Look,




And I modeled a tree too. In there,there are problems in UV seam region. I don't know why it happens.




Between I am reading the sticky thread.

Eric Chadwick

Do you have overlaps in the UVs? You must move all overlaps out of the 0-1 uv square, before baking.

On the tree, it looks like the bake is grabbing a nearby tree limb. A shorter raycast distance, or a better cage, should prevent this.

Michael Knubben

There's some really great information in those stickies that could save you a lot of headache. Untill you really understand normalmaps, there's not much use in helping you solve each of these issues, as they might happen again and again. If you've read them and you're still left with questions, ask them in the stickies so that anyone else reading them can benefit from whatever answer you get!

Also, it would help to see your cage and mesh in xnormal, to see if anything's intersecting or twisted. Once you understand the purpose of the cage, this will become easier as well.