Topogun 2 and xNormal Mapping Problem

Trixx

So this is sort of a 2 part question, but I have a picture that I hope will help make it tolerable.

So I've sculpted my mesh in zBrush, polypainted it, and exported all of the subtools as one .obj into Topogun.

I then retopologized my sword here, UV Unwrapped the low poly mesh in Blender, and went to generate the ambient occlusion and normal maps back inside Topogun.

As you can see in the bottom right part of the image, I'm getting a lot of black spot errors. This happened with my Ambient Occlusion map, and the exact same sections had errors and loss of information in the normal map I tried generating in Topogun.

I've tried turning the "Search Distance" up to a higher value in Topogun in hopes that it would pick up the information from the high detail mesh, but to no avail. I've tried both Software and Hardware rendering.

I eventually gave up with Topogun, and went ahead and tried generating my normal map in xNormal. At first it wasn't working very well. I was getting seams in the normal map itself along the UV seams. I'm guessing this had something to do with low anti-aliasing and not stretching the UV island borders out enough.

I've managed to eliminate the seams, but now I've got these weird distortions happening in my normal map. You can see the distortions circled in red, and also in the provided normal map.

You can see that those distortions are not present in my high detail mesh. (the polypainted and basic material zbrush references)

I've also included the normal map and AO map to give you guys an idea.

My questions are, why am I getting these strange normal map distortions, and why am I getting the strange black holes in my AO map and Normal map in Topogun, despite my efforts to increase the search distance?

Thanks in advanced! I hope one of you veterans can save the day for me. I'm at my wits end.

asyrvachev

Trixx,

not a veteran, but perhaps the issue with AO can be fixed by enabling 'ignore backface hits' in xNormal.

jfitch

It looks like you need some hard edges where your UV seams are around the edge of the blade. Are you exploding when you bake and using a cage?

Trixx

asyrvachev wrote: »

Trixx,

not a veteran, but perhaps the issue with AO can be fixed by enabling 'ignore backface hits' in xNormal.

I'll give that a shot when I get home. Thanks!

Trixx

jfitch wrote: »

It looks like you need some hard edges where your UV seams are around the edge of the blade. Are you exploding when you bake and using a cage?

Sorry, I'm pretty new to this. I'm not sure what you mean by hard edges. Are you referring to using smoothing groups, or adding in extra geometry around the edges in topogun? I'm using blender so the only way to get a hard edge with smoothing is to split the geometry itself, and I think that will mess up my normals.

I'm also not sure what you mean by exploding, and my knowledge of what the cage does is limited. I just loaded my scene with the low poly mesh, and then it asked me for the high detail reference mesh and I loaded that in, and I bake with mostly default settings (aside from resolution and detail quality)

I did notice a black "cage" that would expand when I increased the cage distance value. I increased that to envelope my high detail mesh, and it removed the black holes in the bottom right of my AO map, but I think I remember it causes some other strange errors for me. I'll have to double check that when I get home. In the meantime, do you have any advice as to what I should do to fix this? I didn't import a cage mesh btw; I'm guessing it generated one from my low poly mesh.

AdvisableRobin

Watch/Read this to learn about hard edges/smoothing groups.

Exploding a mesh is a method in which you take the sub-parts that make up the mesh and 'explode' them so that there is space in between each sub-part of the whole mesh. You do this when you have intersecting or very closely adjacent sub-parts of a mesh.

When you bake a map it essentially just shoots a bunch of rays at the surface of the highpoly and sees how it reflects. The cage is basically a way of setting a 'distance' in which the rays will be cast from, anything within the cage will receive the rays.

What I would do. First off, save a copy of everything. Explode your meshes, highpoly and lowpoly, triangulate your lowpoly(Make sure to save a non-triangulated exploded mesh incase you need to edit later), create a cage which fully encloses each piece of your mesh. Bake your Normal and AO maps. Then if you want to get a better AO result, bake a non-exploded AO map using only your lowpoly, which will give you the inter-mesh shadows. You can then combine these two AO maps in Photoshop or whatever image editor you use.

Trixx

jfitch wrote: »

It looks like you need some hard edges where your UV seams are around the edge of the blade. Are you exploding when you bake and using a cage?

Sorry, I'm pretty new to this. I'm not sure what you mean by hard edges. Are you referring to using smoothing groups, or adding in extra geometry around the edges in topogun? I'm using blender so the only way to get a hard edge with smoothing is to split the geometry itself, and I think that will mess up my normals.

I'm also not sure what you mean by exploding, and my knowledge of what the cage does is limited. I just loaded my scene with the low poly mesh, and then it asked me for the high detail reference mesh and I loaded that in, and I bake with mostly default settings (aside from resolution and detail quality)

I did notice a black "cage" that would expand when I increased the cage distance value. I increased that to envelope my high detail mesh, and it removed the black holes in the bottom right of my AO map, but I think I remember it causes some other strange errors for me. I'll have to double check that when I get home. In the meantime, do you have any advice as to what I should do to fix this? I didn't import a cage mesh btw; I'm guessing it generated one from my low poly mesh.

Trixx

AdvisableRobin wrote: »

Watch/Read this to learn about hard edges/smoothing groups.

Exploding a mesh is a method in which you take the sub-parts that make up the mesh and 'explode' them so that there is space in between each sub-part of the whole mesh. You do this when you have intersecting or very closely adjacent sub-parts of a mesh.

When you bake a map it essentially just shoots a bunch of rays at the surface of the highpoly and sees how it reflects. The cage is basically a way of setting a 'distance' in which the rays will be cast from, anything within the cage will receive the rays.

What I would do. First off, save a copy of everything. Explode your meshes, highpoly and lowpoly, triangulate your lowpoly(Make sure to save a non-triangulated exploded mesh incase you need to edit later), create a cage which fully encloses each piece of your mesh. Bake your Normal and AO maps. Then if you want to get a better AO result, bake a non-exploded AO map using only your lowpoly, which will give you the inter-mesh shadows. You can then combine these two AO maps in Photoshop or whatever image editor you use.

Ok, thanks for the tips! I'll try these when I get home. So could the lack of triangulation in my low poly mesh be part of the reason I am getting these distortions in my normal map? It's all in quads at the moment.

AdvisableRobin

Not triangulating before you bake can lead to issues. If you don't triangulate within the 3D app then there is a possibility that the mesh baker and the rendering engine might triangulate it differently.

The errors you are getting at the edge of the blade are most likely due to not using a smoothing split. RE. hard edges or smoothing groups.

Trixx

Alright guys, I've figured it out. I added smoothing groups to make hard edges along the UV seams, and that fixed my distortion, and I increased the cage size for my AO bake in Topogun and it removed the black clipping holes. Thread closed!