Xnormal Cage baking

Acute

Hello everyone, I'm somewhat new to the game asset creation pipeline and was wondering if anyone had any tips when it comes to baking normal maps in Xnormal from hi-poly character/creature meshes.

I was aiming for decently low rez with my LP which means when viewing the meshes on top of each-other there's a good bit of the high poly that is peaking out through it, meaning I need to extrude my cage out by a large amount to fully encompass everything. This though then makes problems with cage overlap in certain areas.

I was testing bakes out with this creature's head and I can get okay results if I spend a lot of time fiddling with the cage, but I'm still getting artifacts. Extruding it without any editing is obviously worse.

Xnormal's controls for cage editing are kind of limited, is there a better program for making cages (in maya maybe?) and if so what's the best method? Or am I doing something else wrong?

Was also wondering if you can export the cage out of Xnormal to take it elsewhere for further editing.

Thanks in advance

Bartalon

The first one looks good to me as far as a bake goes, but I do see some artifacts that may stem from smoothing group issues.

Sometimes it's necessary to get in there and tweak individual components of the cage so every part of your HP is encapsulated. But if you don't intend on changing your low poly's topology then you only have to do it once, even if you need to go back later and move the UVs around.

Personally I don't use xNormal to generate my cage. Instead I use Maya's Transform Component tool.

Boozebeard

You can load in a separate mesh as a cage in Xnormal. So in your model program you can duplicate your LP and edit it to encapsulate the whole HP then export that mesh and load it into xnormal as your cage.

I dunno about exporting the cage you setup in xnormal though. Would assume not.

Acute

Could you tell me a little more about smoothing groups? I wasn't aware they influenced how maps bake.

Haven't used the transform component tool before, does it work similarly or is it simply just pushing and pulling verts around?

Bartalon

Read up on EQ's post here: http://www.polycount.com/forum/showthread.php?t=107196

In most cases, smoothing groups (or hard/soft edges in Maya) should match your UV islands. This means all the edges on the inside of your UVs should be soft, and all the edges along a UV seam should be hard. There are exceptions, but I think if you are just starting to understand their role in baking normal maps, it's a good rule of thumb to follow.

Acute

Thanks, that's very useful. Though if I'm using Xnormal and a cage it sounds like that would be using an averaged projection mesh which wouldn't pick up on any differences when using hard edges or not. At least that's what it seems like he's saying here> "When you use an “Averaged projection mesh” mesh, the use of hard edges is completely irrelevant to the baked end result. Both methods will look exactly the same."

Bartalon

If your UVs don't match up with your hard edges, you will get bake errors. The handplane video should cover at about 3:40 when he starts using the cube as an example.

EarthQuake

Acute wrote: »

"When you use an “Averaged projection mesh” mesh, the use of hard edges is completely irrelevant to the baked end result. Both methods will look exactly the same."

This refers specifically to the normal map projection (the angle at which the rays are cast). As other say, if you have a smoothing group/hard edge, you need to split your uvs there as well, as covered in the hard edge thread which is stickied.