Xamn you xNormal...! Baking problems, would love some help

GarageBay9

Trying to bake a wall piece with some minor protruding geometry. It's not perfect, but I'm getting gradient artifacts around the edges that simply are NOT there in the normals of either the low OR the high poly meshes. It's like xnormal's just making it up. :poly127:

Link to actual meshes if anybody can lend me a hand... http://www.garagebay9.com/storage/airlock.rar

timwiese

I looked at the file and notice you have separate smoothing groups on the outer edge but it isn't split in the UV's. Split your UV's where two different smoothing groups meet (aka on the hard edges) and give it a few extra pixels after you split.

GarageBay9

I tried that but it didn't seem to fix it... Let me give it another shot. Thanks for the help, Tim.

Numerator

GarageBay9 wrote: »

I tried that but it didn't seem to fix it... Let me give it another shot. Thanks for the help, Tim.

Remember what a tangent space normal is. It just represents the normal from the high resolution mesh rotated into the tangent space. It does not matter what the normal map looks like, it matters what the lighting result is.

timwiese

Numerator wrote: »

Remember what a tangent space normal is. It just represents the normal from the high resolution mesh rotated into the tangent space. It does not matter what the normal map looks like, it matters what the lighting result is.

I think you are correct in theory but results such as those on a flat plane will usually show errors when imported into most game engines. This is why tricks such as splitting UV's where there are hard edges are important.

Although the new hotfix for max 2011 works as you describe (as well as the 3point shader) and none of those little errors matter. Most game engines will display something like that as shading errors.

pior

As a matter of fact, artefacts only appear in game engines with a tangent calculation improperly synced with the tool used to bake the maps with.

So the question is : you bake in Xnormal, but what is your target platform/engine/renderer ?

timwiese

pior wrote: »

As a matter of fact, artefacts only appear in game engines with a tangent calculation improperly synced with the tool used to bake the maps with.

So the question is : you bake in Xnormal, but what is your target platform/engine/renderer ?

Hmmm this is interesting to me as I have never worked with an engine that was synched with a certain baking program (or at least it was never documented or trained).
I have always experienced errors such as that as shading errors. Even when viewing models in the Max viewport I could clearly see the errors pop up. Even when baked with max scanline (the exception being the new hotfix in 2011).
So is max viewport not synched with its own scanline normal map bakes?

All these problems is why I just play it safe and use UV splits and smoothing groups to keep my normal maps working with everything.

But I am curious as to how to figure out what programs synch together.

GarageBay9

pior wrote: »

As a matter of fact, artefacts only appear in game engines with a tangent calculation improperly synced with the tool used to bake the maps with.

So the question is : you bake in Xnormal, but what is your target platform/engine/renderer ?

Baking in xNormal, importing to UDK.

Strangely enough, I got it looking correct by going back to the low-poly mesh, unsmoothing everything that was smoothed... and smoothing everything that was unsmoothed. Looks pretty much just the way I wanted it to originally. :poly105:

pior

Haha good then, its all working then!
As for XNormal to UDK, I would expect this setup to work fine for organic shapes but not so well for mechanical objects (even the pipeline Epic used on UT3 was faulty, they had noticeable seam errors suggesting bad syncing)

Grab the exemple meshes from the "why you shouldn't use ..." thread and make a few tests, its pretty enlightening.

P

GarageBay9

Dammit, it's doing it again, and I made sure to split and pad UVs at sharp edges. It's creating awful smoothing artifacts at those junctions as if I hadn't actually split and padded those seams.

If only xNormal's cage projection was a bit more intuitive... it's got the clumsiest UI for cage editing I've ever seen.

GarageBay9

It's rendering wrong in xNormal right now, forget UDK. It's creating these phantom tangent gradients right in the generated normal map as it renders...

Scruples

I dont get it, whats the problem?, the details you claim to not exist are in your high-poly.

[Edit] ah, I thought you meant the high frequency details, not the tangent gradients.

Make sure in Xnormal under low poly meshes under normals is set to 'Use Exported Normals'.
Highpoly normals override tangent space should also be checked.