Home Technical Talk

Problems with baking high to low poly between marmoset 4 and substance painter

Hi, I’ve made an optimized model from a high one using the least ammount of triangles possible and using UDIMS. Before exporting it I’ve triangulated it and exported as a fbx
The baking were made under Marmoset Toolbag 4, flipping the normals in the Y axis.
Some fixes were needed to be made, after fixing It looks with not artifacts at all
imagen
imagen
After that I’ve exported the texture set with correspondent names, when I’m checking if everything is fine, I’ve stumbled with artifacts that are not seen in marmoset.

imagenimagen
By the way, the mesh and UV mapping it’s the following, the retopology it’s handmade:
imagen
imagen
imagen
I'm using blender and exporting as FBX


Replies

  • gnoop
    Offline / Send Message
    gnoop sublime tool
    You low poly model  have same shading gradients. I would try to get rid of them first  because  whatever perfect it may look in  Substance  it still might be there in actual game engine due to normal map compression , screen space something etc .    At least I saw them in Unreal4  , haven't tried v5 much.        Having  low res model without diagonal /corner shading gradients  is a universal remedy.
    Clever hard edge placement / face weighted normals and  using data transfer  to borrow normals from  earlier mesh stages  or from hires  could help to reach it.    Also check in keep normals  in triangulate modifier in Blender.  


       
     Then  try to check in  tangent space  in blender fbx export.   it should  export tangent and binormal vectors too  and  uncheck in  substance import  accordingly  the way Substance wouldn't recalculate them on its own.     Or otherwise let  Substance calculate them i.e  check in in Substance    vs Blender default  export  having tangent space unchecked .          Just a guess  at this point.   To be honest I didn't get why you Y flipped normals  but I haven't used Marmoset for ages.

     
  • Eric Chadwick
  • Fabi_G
    Offline / Send Message
    Fabi_G high dynamic range
    Hi! Don't understand why you flipped the y-channel either, as Painter-Projects can be set to either OpenGl or DirectX. What's your Painter-Project set to? If you flipped the y-channel of Toolbags Normal Map, I'd assume it's DirectX.

    You could bake in Painter see if it comes out any different, if so compare maps.

    Is the lowpoly shaded all smooth? If so, you could harden edges at steep angles that have UV splits anyways, to reduce the lowpolys shading gradients (as gnoop wrote).
  • HAWK12HT
    Offline / Send Message
    HAWK12HT polycounter lvl 13
    Not sure why you have done hard low poly optimization but using UDIM. If its a movie asset you can use way more polygons and get rid of those shading artifacts. For games UDIMs are still not very common workflow. 

    It looks like you have not split smoothing groups where you have UV splits on hard edges like trigger area causing shading artifacts. 

    https://www.youtube.com/watch?v=ciXTyOOnBZQ&t=110s

     
  • illumisanic
    gnoop said:
    You low poly model  have same shading gradients. I would try to get rid of them first  because  whatever perfect it may look in  Substance  it still might be there in actual game engine due to normal map compression , screen space something etc .    At least I saw them in Unreal4  , haven't tried v5 much.        Having  low res model without diagonal /corner shading gradients  is a universal remedy.
    Clever hard edge placement / face weighted normals and  using data transfer  to borrow normals from  earlier mesh stages  or from hires  could help to reach it.    Also check in keep normals  in triangulate modifier in Blender.  


       
     Then  try to check in  tangent space  in blender fbx export.   it should  export tangent and binormal vectors too  and  uncheck in  substance import  accordingly  the way Substance wouldn't recalculate them on its own.     Or otherwise let  Substance calculate them i.e  check in in Substance    vs Blender default  export  having tangent space unchecked .          Just a guess  at this point.   To be honest I didn't get why you Y flipped normals  but I haven't used Marmoset for ages.

     
    Thanks, it looks that's the way, I'm having troubles with parts like the "trigger zone", but I'll be working on it


  • illumisanic
    We have a bunch of tips here that might help you
    http://wiki.polycount.com/wiki/Normal_Map_Modeling

    Oh my God I've should learned that before starting
  • illumisanic
    Fabi_G said:
    Hi! Don't understand why you flipped the y-channel either, as Painter-Projects can be set to either OpenGl or DirectX. What's your Painter-Project set to? If you flipped the y-channel of Toolbags Normal Map, I'd assume it's DirectX.

    You could bake in Painter see if it comes out any different, if so compare maps.

    Is the lowpoly shaded all smooth? If so, you could harden edges at steep angles that have UV splits anyways, to reduce the lowpolys shading gradients (as gnoop wrote).

    I've flipped the Y Channel for DirectX.
    With "harden edges" are you talking about Mark Sharp?
  • illumisanic
    HAWK12HT said:
    Not sure why you have done hard low poly optimization but using UDIM. If its a movie asset you can use way more polygons and get rid of those shading artifacts. For games UDIMs are still not very common workflow. 

    It looks like you have not split smoothing groups where you have UV splits on hard edges like trigger area causing shading artifacts. 

    https://www.youtube.com/watch?v=ciXTyOOnBZQ&t=110s

     

    I've never made a extreme low resolution model of a high one, it's a practice.
    What is "split smoothing groups"?
    So, I should mark seam along hard edges, like here:
  • illumisanic
  • Eric Chadwick
    I've never made a extreme low resolution model of a high one, it's a practice.
    What is "split smoothing groups"?
    So, I should mark seam along hard edges, like here:
    "split smoothing groups" = make hard edges, by splitting the vertex normals.
    http://wiki.polycount.com/wiki/Normal_Map_Modeling#Smoothing_Groups_.26_Hard_Edges

    You could also use Face Weighted Normals, where the model has bevels.
    http://wiki.polycount.com/wiki/Face_weighted_normals

    Generally you want to avoid shading errors in the lowpoly model, because these often cause extreme gradients in a baked normal map, which are bad, see: http://wiki.polycount.com/wiki/Normal_Map_Modeling#Smoothing_Groups_.26_Hard_Edges:~:text=When a mesh,to counteract it
  • illumisanic
    Thanks a lot everyone, it's my first post in this site and I'm glad to see how good this community is
  • illumisanic

    If someone stumbles with this post, In brief, the problem is being solved by correcting shading artifacts, using the Weighted Normals and Data Transfer modifiers. Also marking sharp edges in strategic points, it's about trial and error






  • illumisanic
    I don't know what is happening, but now there is a curve here, in both sides of the model

    I've checked the material uv's and nothing is overlaping, also it's not a triangulation problem


  • Eric Chadwick
    If you share the model in FBX or OBJ format, others might help by examining it for errors. 

    (to upload a model, put it into a ZIP file, then drag-and-drop into your reply)
  • illumisanic

    By the way, I'm using one material per UDIM tile because Marmoset can't accept them
  • pior
    Offline / Send Message
    pior grand marshal polycounter
    First of all congratulations for sticking to it and approaching things carefully, you'll definitely get there. However there are many things you still need to adress.

    - Ditch the UDIMS. Admittedly they are irrlevant to the problems you still need to wrestle with, but anything unnecessary will just get in the way. Just put everything in one well-organized square sheet once and for all.
    - Put as much care in your UVs as you put in your modeling. Treating them as an afterthought like you're doing so far is only getting in the way of a clean bake.
    - Knowing how to model-for-UVs, how to model-for-baking, and how to UV-for-baking are skills not aquired overnight. IMHO you're biting off more than you can chew with this asset in its current state. Perhaps just focus on one part until you get it just right.
    - Sharing your files is necessary for people to help you out. But you'll also need to share your high, otherwise no test bakes can be done.
    - You have a lot of work still ahead of you modeling-wise as the asset is just not looking clean at the moment with poor shading, uneven polygon distribution and improper UVs. Try to find an equivalent asset from a game with similar specs for reference - Fortnite weapons are probably a good fit.




    Whereas on yours the shading issues are immediately apparent :



    Of course knowing how to do things cleanly may seem subjective when you're not used to it yet. So to get there you'll just have to share clean screeshots of your progress for poeple to point things out, little by litte. Baking comes later.


  • illumisanic
    You're right about that, It's the first time I try to do something this level.
    I appreciate a lot the speech, I'll be taking another approach. 
    Maybe I should leave this to rest and practice with more assets Like you said, It's a long way ahead, and I'm getting stuck at something I've first tried
    Here's the high model btw

  • pior
    Offline / Send Message
    pior grand marshal polycounter
    While the high looks decent, in my opinion it is not quite hitting the mark with how thin and tiny many things are. By that I am not saying that they should be stylized or exaggerated but rather that they should be making sense in terms of scale, thickness and roundness for things to look like actual sheet metal and/or machined steel, in order for the surfaces to catch light like they should. This only comes with knowledge of IRL hardware in general and firearms in particular.

    Here is what you have at the moment (high, low wireframe, low shaded) :



    This valve for instance just look way too tiny and easy to break. Same for the connection of the blade to what looks like ... a pressurized gas tank ? Of course these are comments on the design itself rather than the technicalities of baking but things like that pile up. Had this been based on a real item these inconsistencies wouldn't have been as much of an issue. In a way the visual design itself has consequences all the way down to the baking process.



    In production a highpoly like this would receive an extensive paintover pass indicating where to thicken things and what to adjust.

    On the low you are shooting yourself in the foot with all these interpenetrating pieces serving no purpose. This will lead not only to an unnecessary increased workload but more importantly to discontinuities in the textures, causing issues for baking but also for texturing/weathering. And overall there are many inconsistencies with how various edges are treated.



    Personally I think focusing on just 3 parts and getting them right could be a good point of entry : building 3 clean and continuous lows, 3 well-crafted corresponding highs, all of that gathered on a solid UVs sheet treated as if it was a real asset.


  • Eric Chadwick
    Always love Pior crits.
  • pior
    Offline / Send Message
    pior grand marshal polycounter
  • illumisanic
    Honestly the design was more stylish than practical. Fashion over practicity
    Here's another try on only tree parts of the model
    Looking some fortnite models I see that parts of the mesh are directly separated, so I've marked sharp edges on the creases of the model and other parts. Is it wrong? It fixed a lot of issues



  • pior
    Offline / Send Message
    pior grand marshal polycounter
    Well ... if your bake doesn't shade like the high, then it's wrong. If it shades exactly like the high, it's right.

    Sure enough this dark area on the dagger shouldn't be there : 


Sign In or Register to comment.