Home Technical Talk

issue in marmoset when using normal map

vempdance
polycounter lvl 5
Offline / Send Message
vempdance polycounter lvl 5
Hi everyone,I am rendering a model for my portfolio in marmoset toolbag.But when I use normal map I find a promblem like this:
problem2.JPG?token_hash=AAFrZqp-2xaQb8zy6AZyxXDTjv_DHYbbDSNU0AMTkDlWow&dl=1
You can see there is some hard edges in the chest.

And then I put the same model and the same normal map into maya.The render result is like this:
problem1.JPG?token_hash=AAGKLhfRXeMnQxyvV31l8ryDGPvtptVw58oigN21XjPzDw&dl=1
It seems every thing is all right.

The low-res model is made in maya and I make all the egdes into soft edge before baking.The normal map is baked in xnormal,and the body's parts is like this:
normal11.JPG?token_hash=AAFrZqp-2xaQb8zy6AZyxXDTjv_DHYbbDSNU0AMTkDlWow&dl=1

Could anyone help me deal with this? Thanks very much and I appologise with my bad English.

Replies

  • Bartalon
    Options
    Offline / Send Message
    Bartalon polycounter lvl 12
    vempdance wrote: »
    The low-res model is made in maya and I make all the egdes into soft edge before baking.

    Your low poly should have all UV border edges set as hard edges. This should resolve your artifacting, provided you are using a cage.

    You should probably grab this MEL script from Jon Stewart. It automatically hardens UV borders.

    http://jonathonstewart.blogspot.com/search/label/Scripts
  • vempdance
    Options
    Offline / Send Message
    vempdance polycounter lvl 5
    Bartalon wrote: »
    Your low poly should have all UV border edges set as hard edges. This should resolve your artifacting, provided you are using a cage.

    I tryed that but it doesn't work.For cage,I just duplicated the low poly and imported it into xnormal as a cage(of course adjusted the vetex to make it be a cage),and the cage is all soft edges. Is this responsible for the problem?

    I just found that the second picture's link is broken so I repaired it.Now you can see the same low poly and the same normal map worked well in maya.

    and thanks for the script,it's very useful:poly124:
  • BARDLER
    Options
    Offline / Send Message
    BARDLER polycounter lvl 12
    Bartalon wrote: »
    Your low poly should have all UV border edges set as hard edges. This should resolve your artifacting, provided you are using a cage.

    You should probably grab this MEL script from Jon Stewart. It automatically hardens UV borders.

    http://jonathonstewart.blogspot.com/search/label/Scripts

    That is not really true. You do not need to have your UV border edges set to hard edges in order to avoid artifacts. You need to have hard edges set on edges that have harsh angles approaching 90 degrees, and where ever you put those hard edges, you need a UV split. On organic models, like characters, rocks, ect, you can almost always get away with a single smoothing group because they don't often have harsh angles. This video has a lot of good info in it [ame="http://www.youtube.com/watch?v=ciXTyOOnBZQ"]Controlling Shading Behavior - YouTube[/ame].

    With that said, there really isn't a good reason NOT do set all your border edges to hard regardless if its needed or not. Simply saying that you have to set all your UV border edges to hard inst really a good explanation of how to avoid artifacts.

    As for the OP. It looks like that line is following the edge of your UVs, Is that true? Did you do anything to change the vertex normals of your mesh after you baked it? Weld anything, set normal angle, delete edges, anything like that? What file format are you saving out from in xnormal, and are you doing anything in photoshop to it? All those things can cause issues.

    As for your cage question. Yes you want your cage to have averaged normals.
  • vempdance
    Options
    Offline / Send Message
    vempdance polycounter lvl 5
    BARDLER wrote: »
    That is not really true. You do not need to have your UV border edges set to hard edges in order to avoid artifacts.

    Thanks very much for the reply. I learned a lot from that.

    This is the UV boarded in this area:
    board1.JPG?token_hash=AAEFLXb-DtKuKTNObdmKM_xCuUXVAq53feB-n6ykLuk9cA&dl=1

    So it is not all follow the UV boared.And the problem is just happened in this little area, all the other parts of the model looks well, so I don't think it has anything to do with UV borads.

    And I didn't do anything to the model after I prepaired to bake.The map baked out with xnormal is .tga file, and I just combined them in photoshop.
  • Bartalon
    Options
    Offline / Send Message
    Bartalon polycounter lvl 12
    BARDLER wrote: »
    With that said, there really isn't a good reason NOT do set all your border edges to hard regardless if its needed or not. Simply saying that you have to set all your UV border edges to hard inst really a good explanation of how to avoid artifacts.

    Ah, yeah. My response was poorly worded and poorly explained. I should have linked that video. :) Generally my two rules of thumb are "UV Seam at ~90 degree edges" and "If it's a UV seam, make it hard-edged." It's not a catch-all but since I started following that I haven't had many issues, even across spherical surfaces.
  • Bek
    Options
    Offline / Send Message
    Bek interpolator
    BARDLER wrote: »
    With that said, there really isn't a good reason NOT do set all your border edges to hard regardless if its needed or not.

    Sure there is. Cylinders for one. Or organic shapes, like in this case a character.

    You should have a UV seam where you have a hard edge
    This does not mean you must have a hard edge where you have a UV seam.

    Whether you'll notice the difference on any given model is another matter.
  • vempdance
    Options
    Offline / Send Message
    vempdance polycounter lvl 5
    I'm sorry to post again but I really want to solve this problem.Could anyone help me?
    Infact I have the same problem in another model.Maybe I should just give up marmoset...
  • ElleKitty
    Options
    Offline / Send Message
    ElleKitty polycounter lvl 3
    I cant shake off the feeling that this has everything to do with Marmoset's relation to smoothing groups. Whenever your nmap is being generated in xnormal and Max, the lowpoly's own smoothing is taken into account; the nmap is crafted to the lowpoly smoothing that is there at the time of render. I've never used Marmoset, but chances are that it does not respect the smoothing in the same way that Max/Maya/xnormal do, if at all.

    The two things that you could try are...
    1. Edit your lowpoly model in Maya. Try setting the smoothing to the whole model and generate the nmap in xnormal, and then also try generating one more without any lowpoly smoothing at all (facets). See how Marmoset likes each of the two nmaps. In xnormal you would use the "Use Exported Normals" option in the lowpoly section of xnormal.
    2. Alternately, use "Average Normals" and "Harden Normals" to achieve the same without having to edit the model in Maya beforehand. These two options will output different nmaps, one for a fully smoothed model and one for a fully facetted model. one of them should do it.
  • AlecMoody
    Options
    Offline / Send Message
    AlecMoody ngon master
    According to EQ, marmoset imports normals and attempts to import tangents. You might try handplane's "input tangent and binormals" output. That output looks at what is in the fbx file and generates the normal map based on that info- much like marmoset will be doing when it renders the normal map.


    I have found that xnormal has issues with baking the tangents inside the mesh file. I haven't tested that for about a year but in the past it hasn't produced the result you would want.
  • AlecMoody
    Options
    Offline / Send Message
    AlecMoody ngon master
    I did a quick test in the toolbag with that output. The results are 99% correct. There are some very subtle shading errors in a few places but this is very good considering the object (which is smoothed):

    61aa2acb9bd01abe125c1ee18cf8a72b.png

    Without the normal map:
    d27fcd3f1716c34586b500e9b03e9602.png

    Make sure your fbx has tangents and binormals exported when you make the tangent space map and when you load it into marmoset.
  • vempdance
    Options
    Offline / Send Message
    vempdance polycounter lvl 5
    AlecMoody wrote: »
    According to EQ, marmoset imports normals and attempts to import tangents. You might try handplane's "input tangent and binormals" output. That output looks at what is in the fbx file and generates the normal map based on that info- much like marmoset will be doing when it renders the normal map.


    I have found that xnormal has issues with baking the tangents inside the mesh file. I haven't tested that for about a year but in the past it hasn't produced the result you would want.

    It works! I exported a fbx file, baked a object space nmap,then used handplane's output that you told me and get a result like this:
    marmo.JPG?token_hash=AAHtMZMkPQz2GqYU-p0p7iDGWu0PTTkpTaVFGpSeUKNqFw&dl=1

    Thank you very much for even do a test for me!:)

    By the way, the same fbx file and nmap now have the same issue in maya... Is that means I must use diffrent types of 3Dmesh and maps in diffrent 3D applications? Sorry for this simple question but I'm a rookie in CG so I really botherd by those technical issues... :poly101:
  • AlecMoody
    Options
    Offline / Send Message
    AlecMoody ngon master
    Getting a normal map to render correctly is a combination of a lot of factors. Generally speaking, what works in one game engine or 3d modeling tool will not work in another. If you want to make a copy of the normal map that works in maya use one of the maya outputs from handplane.

    There is also a ton of information covering normal mapping and baking on the handplane youtube channel. If you are really interested in better understanding what is happening with your normal maps, watch all of the videos in the order they were uploaded:
    www.youtube.com/handplane3d
  • AlecMoody
    Options
    Offline / Send Message
    AlecMoody ngon master
    Oh also, there is still some kind of artifact in there. It looks like the triangulation direction needs fixing. Try force triangulating that quad, then making a new tangent space map, and then bringing that object into marmoset.
  • vempdance
    Options
    Offline / Send Message
    vempdance polycounter lvl 5
    AlecMoody wrote: »
    Getting a normal map to render correctly is a combination of a lot of factors. Generally speaking, what works in one game engine or 3d modeling tool will not work in another. If you want to make a copy of the normal map that works in maya use one of the maya outputs from handplane.

    There is also a ton of information covering normal mapping and baking on the handplane youtube channel. If you are really interested in better understanding what is happening with your normal maps, watch all of the videos in the order they were uploaded:
    www.youtube.com/handplane3d

    Thanks very much. I've learned a lot from your replies. The videos are very helpful and I didn't find that Handplane is such a great tool before:poly124:.Works beccome easier and have a lot of fun because of guys like you.Thanks again!:thumbup:

    And also thanks to all who's tring to help me.You have my best wishes!:D
Sign In or Register to comment.