Home Technical Talk

Normal Map Baking Problems (xNormal)

polycounter lvl 10
Offline / Send Message
Benvox2 polycounter lvl 10
Hey all,

I have been working on a cannon for a pirate ship and so far I have; created the low poly in max, given it a UV unwrap, taken it across into Zbrush and given it a high poly sculpt version, but now when im trying to combine the low poly and high poly in xNormal im getting strange errors in the normal map bakes.

anuddatest_normals.png

I have divided up the model and import the pieces separately as opposed to one giant mesh, but these are the results I get on this particular piece and I can seem to find the source of the problem. I suspect it could be something to do with my Unwrap, my exporting options from either max or zbrush, or something to do with the pivot or placing in the world of both meshes? They seem to sit perfectly on top of each other in the max viewports. Apart from the obvious artefacts the whole map seems very harsh with those pink sections also, any help would be very much appreciated.

Thank you

Replies

  • rollin
    Offline / Send Message
    rollin polycounter
    so what is your problem.. the bake for itself looks normal .. as long as we don't see more images of the high the low with and without normalmap

    (just a little guess.. ray distance?)
  • Benvox2
    Offline / Send Message
    Benvox2 polycounter lvl 10
    rollin wrote: »
    so what is your problem.. the bake for itself looks normal .. as long as we don't see more images of the high the low with and without normalmap

    (just a little guess.. ray distance?)

    Here's a shot of the high poly, the low poly and both combined in 3ds max. I have tried several different timings for the ray distance calculator within xNormal from 5 sec to well over a minute with no change in results, its almost as if the normal baking is too harsh and strong if that makes sense?

    Im also concerned with those harsh contrasted areas of pink showing up in the original normal map shot.

    views.jpg
  • rollin
    Offline / Send Message
    rollin polycounter
    no setting ray distance by yourself .. not timed

    could you just make a quick bake with a cage?
  • Bartalon
    Offline / Send Message
    Bartalon polycounter lvl 12
    Try breaking those harsh edges into their own smoothing groups/UV islands and give them some space between each other. If you would consider a point where two surfaces meet a "corner," it should probably have a UV split.
  • Benvox2
    Offline / Send Message
    Benvox2 polycounter lvl 10
    rollin wrote: »
    no setting ray distance by yourself .. not timed

    could you just make a quick bake with a cage?

    Do you mean make my own cage in max? I thought xNormal kinda took care of that.. Thanks for the help btw!
  • Benvox2
    Offline / Send Message
    Benvox2 polycounter lvl 10
    Bartalon wrote: »
    Try breaking those harsh edges into their own smoothing groups/UV islands and give them some space between each other. If you would consider a point where two surfaces meet a "corner," it should probably have a UV split.

    Hello, I thought that the low poly mesh should have all smoothing groups cleared? Or be 1 snoothing group? I was also told to try the 'average normals' option in xNormal which doesn't seem to affect my bake..

    Thanks for chiming in!
  • rollin
    Offline / Send Message
    rollin polycounter
    Benvox2 wrote: »
    Do you mean make my own cage in max? I thought xNormal kinda took care of that.. Thanks for the help btw!

    you can create one in max and in xnormal. You have to make sure exporting and enabling it in xnormal though.

    Anyhow, xnormal allows you to set ray distances without cage but you are obviously doing something wrong or it's something else. if you have the exact same result with a cage it could be magic but I guess you just haven't set the correct min / max ray distances for your lowpoly model in xnormal

    also you can bake a ray-miss map to better highlight these areas
  • Bartalon
    Offline / Send Message
    Bartalon polycounter lvl 12
    Benvox2 wrote: »
    Hello, I thought that the low poly mesh should have all smoothing groups cleared? Or be 1 snoothing group? I was also told to try the 'average normals' option in xNormal which doesn't seem to affect my bake..

    Thanks for chiming in!

    Smoothing groups are somewhat dependent on how you plan to view your model in engine. If it's flat lighting, smoothing groups don't matter. However, if you are baking normal maps there are some guidelines you should follow to improve your bakes. There are exceptions, but generally the same steps are followed when preparing a model for normal mapping.

    1) UV islands should each have their own smoothing group assignment so smoothing splits fall on the same edges as your texture seams.

    2) If your model contains harsh corners that are exactly or close to right angles, a smoothing group split is required for proper normal map baking. This is due to the resulting averaged vertex normals trying to bend light across two surfaces that would be realistically impossible. Consider chamfering these edges instead if you can afford it.

    3) If you are forced to create a smoothing group split, also split your UV island to match.

    4) Give your UV islands sufficient breathing room from other UV islands (they shouldn't be touching).

    5) If you can help it, try to keep your UV islands straight. For instance, if you have a rectangular island, it shouldn't be oriented diagonally. This will help mitigate lighting issues along seams.

    I would also recommend creating a cage for your low poly. The cage determines the ray distance. Use a Push modifier to create a cage based on your low poly, and make sure it completely encapsulates your high poly, but don't push much further than necessary or it can cause baking problems.
  • EarthQuake
    You can create a cage within XN as well, just load up the 3d viewer. Should be veru fast for an object like this. If you're going to bother setting up a cage etc in max, why not just bake in max?
  • Twotents
    Offline / Send Message
    Twotents polycounter lvl 4
    I really dislike the baking system in Max, I tend to do most of my baking in Maya or xnormal.

    Just a personal preference though, but give baking a try in Maya it just has less hassle to it.
  • Benvox2
    Offline / Send Message
    Benvox2 polycounter lvl 10
    Bartalon wrote: »
    Smoothing groups are somewhat dependent on how you plan to view your model in engine. If it's flat lighting, smoothing groups don't matter. However, if you are baking normal maps there are some guidelines you should follow to improve your bakes. There are exceptions, but generally the same steps are followed when preparing a model for normal mapping.

    1) UV islands should each have their own smoothing group assignment so smoothing splits fall on the same edges as your texture seams.

    2) If your model contains harsh corners that are exactly or close to right angles, a smoothing group split is required for proper normal map baking. This is due to the resulting averaged vertex normals trying to bend light across two surfaces that would be realistically impossible. Consider chamfering these edges instead if you can afford it.

    3) If you are forced to create a smoothing group split, also split your UV island to match.

    4) Give your UV islands sufficient breathing room from other UV islands (they shouldn't be touching).

    5) If you can help it, try to keep your UV islands straight. For instance, if you have a rectangular island, it shouldn't be oriented diagonally. This will help mitigate lighting issues along seams.

    I would also recommend creating a cage for your low poly. The cage determines the ray distance. Use a Push modifier to create a cage based on your low poly, and make sure it completely encapsulates your high poly, but don't push much further than necessary or it can cause baking problems.

    Ahh thank you! Im getting much cleaner results after separating the hard edge pieces into unique smoothing groups and UV islands. So after this you recommend making my own cage as well? Can I duplicate my low poly in max, then push modifier so it just contains all of the high poly, and then export that and use it in xNormal as the cage?
  • Benvox2
    Offline / Send Message
    Benvox2 polycounter lvl 10
    rollin wrote: »
    you can create one in max and in xnormal. You have to make sure exporting and enabling it in xnormal though.

    Anyhow, xnormal allows you to set ray distances without cage but you are obviously doing something wrong or it's something else. if you have the exact same result with a cage it could be magic but I guess you just haven't set the correct min / max ray distances for your lowpoly model in xnormal

    also you can bake a ray-miss map to better highlight these areas

    So the options are to push out a cage from my low poly in max and import into appropriate field in xNormal or create one in xNormal itself? How do I make one in xNormal?
  • Benvox2
    Offline / Send Message
    Benvox2 polycounter lvl 10
    EarthQuake wrote: »
    You can create a cage within XN as well, just load up the 3d viewer. Should be veru fast for an object like this. If you're going to bother setting up a cage etc in max, why not just bake in max?

    Sorry I don't quite follow, I create the cage in xNormal in the 3d viewer? I keep getting crashes in the viewer because my high poly (straight from zbrush) is too high poly!
  • EarthQuake
    Benvox2 wrote: »
    Sorry I don't quite follow, I create the cage in xNormal in the 3d viewer? I keep getting crashes in the viewer because my high poly (straight from zbrush) is too high poly!

    Yeah, you can create a cage in the XN 3d viewer, the tools to do this are fairly primitive, but for simple meshes its quick to do. Try decimating the mesh, or swapping it for a lower sub-d level export if its crashing (and swap back before you bake)
  • rollin
    Offline / Send Message
    rollin polycounter
    in max the projection modifier allows to set up a cage. when exporting as sbm (what you really should do) you can export that cage to xnormal
  • Benvox2
    Offline / Send Message
    Benvox2 polycounter lvl 10
    Thanks a lot for all the suggestions guys, I've been getting much cleaner results and have a much better understanding on the whole normal map baking voodoo, just have to tinker with my cage's and I should be right, much appreciated! :)
Sign In or Register to comment.