Home Technical Talk

Making sense of hard edges, uvs, normal maps and vertex counts

1679111215

Replies

  • Boonta
    Offline / Send Message
    Boonta polycounter lvl 13
    Cheers Froyok !!
  • Badsearcher
    Ok, so, I really need to know if I comprehend this correctly because I'm absorbing a lot of information that clashes with my pre-conceived notions.

    If I want 100 percent no tears normal maps drawn up by Xnormal I need to :

    1: Use a cage for anything more complex than a single UV Island/Single smoothing group mesh. (Using a cage is non-negotiable? It's not enough to use the average normals function from the smooth normals drop down in the low definition meshes window?)

    2: The low definition mesh used by x-normal needs to be one smoothing group?

    3: In Maya, the low definition mesh MUST have hard edges at all UV seams?

    4: 90 degree corners more or less should be hard edges and separated UVs?"

    5: All hard edges should be separate UV's?

    Thank you all in advance for all the write ups and any help in advance, I just want to know if I have my facts straight.
  • EarthQuake
    Ok, so, I really need to know if I comprehend this correctly because I'm absorbing a lot of information that clashes with my pre-conceived notions.

    If I want 100 percent no tears normal maps drawn up by Xnormal I need to :

    1: Use a cage for anything more complex than a single UV Island/Single smoothing group mesh. (Using a cage is non-negotiable? It's not enough to use the average normals function from the smooth normals drop down in the low definition meshes window?)
    Yes, non-negotiable. If you use average normals you will overide the lowpoly mesh normals and get errors when you apply the nm to your mesh.
    2: The low definition mesh used by x-normal needs to be one smoothing group?
    No, you don't have any restrictions on your lowpoly, this is why its important to use a cage. If you don't use a cage, hard edges on your lowpoly will result in seam errors in the bake.
    3: In Maya, the low definition mesh MUST have hard edges at all UV seams?
    No, it isn't a must, but it is free in terms of performance hit so there is little reason not to.

    OTOH, you MUST have uv splits eveywhere you have hard edges, otherwise you will get a different sort of seam artifact in the bake.
    4: 90 degree corners more or less should be hard edges and separated UVs?"
    There is no hard rule here, it depends on a variety of factors, you should do some test bakes to see what you can get away with in your particular baker/engine. If your engine is synced to your baker, you can get away with more extreme mesh normals before you get smoothing errors.

    But generally yes, 90 degree angles or great is a fine rule of thumb.
    5: All hard edges should be separate UV's?
    Yes.
  • Badsearcher
    Thank you so much for all your help Earthquake. A link to this thread should be displayed when a user first installs Xnormal.
  • Badsearcher
    Ok! So, if I understand correctly.

    1: If I use a cage mesh in Xnormal (a cage mesh which I import from Maya where I define it by hand), it overrides or makes irrelevant the smooth normals setting.

    2: The cage mesh should be set to all soft/averaged edges regardless of the low-poly.

    3: The low-polys hard edges should be defined before baking (Wrong= or should the low-poly edges be averaged out and then in Maya be hardened where needed?)

    4: UV borders can have a "free" hard edge on them.

    5: There MUST be a UV border at all hard edges.

    6: The 90 degree hard edge rule of thumb.

    7: In Xnormal, don't use ray distance calculator because cage mesh overrides.

    Do I understand things correctly enough?

    Also, I have a maya specific question which I couldn't fine a straight answer for. When I use a normal map in Maya and I am in the file node for the texture, what filter type do I choose?

    I see a lot of references to mip maps but I haven't found any way to make Maya use mip maps. Also, one tutorial in passing said to just set filter type to off without explanation. Is that correct?

    Edit: After doing some test bakes, I realize that the normals need to be defined on the low poly mesh I'm inputting into XNormal.
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    I have been experimenting with baking some parts of my car in the past few days with various degree of success. I've decided to make some really low poly mesh and see if it bakes well. And it didn't even though I used a cage and separate UV islands with hardened edges. Is it because it's TOO LOW poly or am I doing something wrong?

    Here is the mesh baked right - with the cage, separate islands and so on.

    alrightn6bm7.png


    Here is the mesh baked with one smoothing group. You can see it looks worse and more seams coming out.

    worsehxx4y.png

    badbakeqwans.png

    It looks even worse if I unwrap it as one UV island but that's the way it should be in this case.
  • EarthQuake
    With a mesh like this, you should be able to get away with no hard edges, and less uv seams really. What are you baking in? I feel like that might be the source of your problems.
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    I baked it in Modo but it's the same in Xnormal... without a cage. Here it's with a cage:

    normalruo1z.png

    But I baked it in Modo with the cage! It's either not working or I made the morph map very wrong. I'm yet to use Farfarer's workaround with transparent cage so may be I'll be able to fix it inside Modo.
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    I have fixed in Modo. I just had to make the mesh smooth again and then reassign smoothing groups. For some reason a script messed it up (or may be it was just me).
  • Badsearcher
    Quick question about Cages/Average Projection meshes.

    Do self-intersections in cages matter?

    Here is an extreme example (it's projected out more than it needs to be, I know) of what I'm talking about where the low-poly cage self-intersects but doesn't intersect the high poly.

    I have to know because I'm doing a bunch of bakes involving crevices and when you "inflate" the cage mesh then a lot of the crevices self-intersect but I figure if XNormal is only baking one face at a time then self-intersection doesn't really matter, right?

    I'm tweaking the cage mesh to avoid self-intersection but I'm concerned that I'm wasting my time AND causing distortion errors in XNormal.
  • EarthQuake
    A small amount of overlap in the cage itself can generally be ok, but I've had issues with extreme overlaps, generally you should model your low and low in such a way that you don't need to extend the cage out very far.
  • Badsearcher
    Huh, that's surprising cause I would expect that any overlap would be bad, I wonder what defines the boundary between acceptable self-intersection and unacceptable self-intersection.

    I'm gonna do some experiments comparing results from different kinds of cages (I'm making some clothing and crevices are nearly unavoidable) since cages with a little bit of overlap aren't that labor intensive.

    As usual, thank you.
  • EarthQuake
    From my experience I think the issue was that at a certain point with the overlap, the normals on the cage mesh are just facing in a wacky direction and you get all sorts of errors.

    But I never really took the time to research this hunch, generally I just avoid it.
  • Badsearcher
    That is so crazy, normal maps have been around since 1998 and there's so much about the process that is undocumented. You're essentially a pioneer in something that so many people use.

    I wish I could find it but I remember reading somewhere where someone high up at Pixologic said, "Eh, don't use normal maps, just use a displacement map topped with a bump map, normal maps will just give you errors," like it was something that he had totally given up on.
  • WarrenM
    I almost get the impression that normal maps were meant to be a stepping stone to the "next gen" ... just a quick hack to get more detail until we could push the amount of polys we wanted. However, years later we're still waiting on that magical rendering solution and we're all still using the "temp" hack. :)
  • Badsearcher
    Quick question. Does it matter if the cage mesh has different uv's than the low-poly mesh?
  • WarrenM
    UVs on the cage mesh are irrelevant.
  • EarthQuake
    WarrenM wrote: »
    UVs on the cage mesh are irrelevant.

    Not sure that is entirely correct, though I'm not sure why you would ever have a cage mesh that doesn't have uvs. You're making the cage mesh from the lowpoly, right? Not some random other mesh?

    Generally, your cage needs to match the vert order exactly, so exporting without uvs is probably going to mess it up in something like Xnormal.

    I'm not really aware of a way to change the uvs on a cage/envelope mesh in Max/Maya.
  • Badsearcher
    I changed the low poly's uvs but transferring uv's to the cage mesh is a matter of a few seconds so I'll just do it.
  • Badsearcher
    Actually, I decided to experiment. I created two topologically identical cage meshes in Maya, one with the old uv's and one with uv's that matched the low poly mesh.

    Now, I didn't examine the normal maps created by xnormal pixel by pixel but the uv's appeared to have made zero difference.

    I can see how maybe this is an important issue because in some apps the vertex order might get modified by a change UV operation but that does not seem to apply to a Maya to Xnormal workflow.
  • EarthQuake
    Actually, I decided to experiment. I created two topologically identical cage meshes in Maya, one with the old uv's and one with uv's that matched the low poly mesh.

    Now, I didn't examine the normal maps created by xnormal pixel by pixel but the uv's appeared to have made zero difference.

    I can see how maybe this is an important issue because in some apps the vertex order might get modified by a change UV operation but that does not seem to apply to a Maya to Xnormal workflow.

    Cool, that's good to know, I hadn't experimented with it so I was just sort of guessing that it may be an issue.
  • WarrenM
    I can see EQs point ... differiing UVs COULD change the order of the verts in some way.
  • emesjay
    Offline / Send Message
    emesjay polycounter lvl 5
    Thanks for the thread EQ.
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    almost two weeks of baking and I've finally figured out hard edges, UV splits, cage baking and so on. Finally it works (at least in xnormal). But damn, I wish this step wouldn't be needed.
  • Badsearcher
    dzibarik wrote: »
    But damn, I wish this step wouldn't be needed.

    Well, I don't know man, now that you know how the sausage is made, maybe you should figure out how to automate it and then leverage your programming for maximum profit.

    *shrug*
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    Well, I don't know man, now that you know how the sausage is made, maybe you should figure out how to automate it and then leverage your programming for maximum profit.

    *shrug*

    yeah, that's what I'm currently doing with various scripts for Modo. I haven't looked into batch baking yet, just exploding mesh/making cage scripts.
  • Bek
    Offline / Send Message
    Bek interpolator
    I hope by that you mean the wonderful PipelineIO script. If you haven't already check it out — huge time saver.
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    Bek wrote: »
    I hope by that you mean the wonderful PipelineIO script. If you haven't already check it out — huge time saver.

    yeah, that's the one. But it has some oddities, like models can't be viewed in Xnormal, it's just a black screen. But then it bakes alright even if I can't view it.

    Still I'd prefer baking in Modo or Substance Designer. Modo produces some unpredictable results, I haven't figured it out yet and Substance Designer has different issues - I described them in the master thread but I don't have time to go into details. Xnormal just works and produces predictable results so I'll use that for the time being.
  • WarrenM
    That's the same here. I WANT to bake in Modo but just can't. I can't get it to behave consistently or predictably ... xNormal works first time, every time.
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    it seems this may be the problem - http://community.thefoundry.co.uk/discussion/topic.aspx?f=83&t=87255

    But world space normals behave unpredictably for me too. Like all technical issues I'll figure it out someday but I don't have time right now.
  • Bek
    Offline / Send Message
    Bek interpolator
    dzibarik wrote: »
    yeah, that's the one. But it has some oddities, like models can't be viewed in Xnormal, it's just a black screen. But then it bakes alright even if I can't view it.

    Works fine for me; I'm guessing the problem might be scale differences (causing your xnormal viewport to be looking at nothing — press the center view button.
  • igi
    Offline / Send Message
    igi polycounter lvl 12
    Bek wrote: »
    I hope by that you mean the wonderful PipelineIO script. If you haven't already check it out — huge time saver.

    I think most of things pipelineIO intended to do is now covered by modo801 version. Store all your surface info in the vertex map, click export selected and here you go :)
  • WarrenM
    Yeah, I'm not sure PipelineIO really matters in an 801 world. Farfarer's vertex normal script is indispensable, however. And yeah, as I said above, just bake in xNormal.
  • Nam.Nguyen
    Offline / Send Message
    Nam.Nguyen polycounter lvl 9
    I just tried "hit only matching material ID" in max for complex mesh, it's pretty useful, but it has a small problem that bug me is after I hit render, my mesh just show 1 material ( the material ID of sub-objects still there, it just not showing )

    RcI3HJV.jpg
  • reedweinstock
    Offline / Send Message
    reedweinstock polycounter lvl 8
    Ignore This Post. I two different tabs open I ment to post this in the "How u model dem shapes thread" :(

    Having trouble figuring this one out.

    3Px2JC8.jpg
    d7ES2wD.jpg

    Thanks,
    Reed
  • Nintendo46
    Hey there guys

    i already learnt a lot of this thread, but im still struggeling with my normal maps...
    So i cant really show of anything here, because it just doesnt look good enough. Really annoying, but i havent found a way out.

    So i modeled this lamp and created a highPoly around it. Then i scaled the lowPoly so its just a little bigger than the highpoly, took care of my smoothing groups (made all edges hard in maya) and then exported and baked a normal in xNormal

    Heres the result im getting in my viewport...

    weirdNormal_003_zps3ad2692f.jpg

    Something with the Projection is just wrong but i could figure out what exactly. I tried everything, different smoothing groups, match the shapes as close as possible, used the ray calculator tool in xNormal, but nothing works... :(

    well here are some other images that show this mess:

    weirdNormal_002_zpsaefc2911.jpg

    weirdNormal_001_zps04b78103.jpg

    weirdNormal_000_zpsf520e7d0.jpg

    Im very thankful for any help, it really sucks because i have loads of ideas to visualize but cant produce them in 3d LOL

    and heres the scene as a fbx if someones interested..

    http://bayfiles.net/file/1gEr6/vTbc0S/weirdNormals.fbx

    So Thanks anyway

    With best regards, Joel
  • Quack!
    Offline / Send Message
    Quack! polycounter lvl 17
    Your lowpoly should be the exact size as your high poly, not larger.
    Triangulate before you bake.
    Set your hard edges to your UV islands.
    Use as few UV islands as possible, the entire left and right side of this object could each be 1 uv island for example.
  • Joopson
    Offline / Send Message
    Joopson quad damage
    You never mentioned using a cage either. Are you?
  • JGreentheRookie
    Offline / Send Message
    JGreentheRookie polycounter lvl 10
    This thread saved myself long hours of painful head pain:poly127:. Thank you very much for existing:) :poly142:

    I used to have only one smooth group on my assets. Most of the time I had no artifacts, everything worked fine and superb until 2days ago when I had a different kind of model.
    But now I know what to do.
  • EarthQuake
    nintendo: you need to use a cage.
  • Nintendo46
    Hey there

    First of all thanks to Quack!, Joopson and Earthquake

    so..... i really have been missing the cage. But i didnt actually forget it, but i didnt "activate" it in xNormal. I was assuming that when i opened the 3DViewer and nothing of the low poly sticks through the cage it would be fine. But i was totally wrong, i set the cage up, changed the uvs and the result looks a lot more like i had in mind :

    howitlooksnow_zps185d1190.jpg

    So this means that even when theres no penetration in the viewer the cage must be setup in xNormal and the Meshes be saved out accordingly?

    Anyways thanks for the tips, but im still unsure about the size of the lowPoly. Doesnt it have to be just a little bigger than the highPoly? Otherwise i could use vertexSnapping when modeling, but im not shure.

    So my personal checklist goes like this:

    - Hard edges on uv borders (im using maya)
    - lowPoly a little bit taller than the highPoly
    - Create UV borders at hard Edges in the model
    - triangulate the lowPolymesh before importing in xNormal
    - add additional geometry for better rayDirection when needed
    - set up a cage for every model and make it a little bigger than the bake mesh in xNormal

    Are the thoughts right? Im not so shure about them..

    And what do you guys think of my normal map now? Does it look good enough?

    normals_atm_zps6a717a0a.jpg

    thanks in advance

    With best regards, Joel
  • EarthQuake
    The lowpoly should match the size as closely to the highpoly as possible (this usually means that it will be a little smaller in some areas, and a little bigger in others). No need for it to be bigger than the high (this is one of the things the cage does).

    highlowshape.jpg

    The rest of your workflow seems sound, though you may want to explode your high/low poly meshes to avoid errors where meshes intersect.
  • Nintendo46
    Thanks for your answer Earthquake

    But i dont understand what you mean by "explode".Make them further away from each other?
  • EarthQuake
    yeah, you move both the lowpoly and high poly elements away from each other (usually with keyframes in your 3d app) to avoid intersection errors, after you bake, move them back (which is why the keyframes are helpful).

    Basic concept explained here:

    https://www.youtube.com/watch?v=bgYoXF6QmWw
  • Nintendo46
    Thanks for your help Earthquake

    i will look into that tutorial and try to implement it.

    thanks, man
  • Sunn
    Offline / Send Message
    Sunn polycounter lvl 4
    Guys hi, I'm new around here. I have a few questions for anyone who is willing to take the time to answer.

    Concerning UV mapping and unwrapping props.

    - Can multiple props share a UV space even if you aren't using all those props in the level or is this wasteful?

    - How are textures made to repeat/tile if you are unwrapping.

    - How do you set up multiple UVs for lightmaps in UE4 (sorry I've looked and looked) do you have to do a whole new layout?

    - Whats one tip you wish you knew when you first started 3d, when you first started doing UV?

    - Is it ok to have your hands in every aspect from modelling, texturing, animating and coding, or is that too much for one person?

    Thanks for your times? Hope this is the proper thread for this.
  • final_fight
    Offline / Send Message
    final_fight polycounter lvl 9
    Hard edges or not I'm having visible seams anyway. What could be the problem?

    image.png
    image.png
    LP2_Normals_Map.png
  • Nintendo46
    Hi there

    Ive created this small little PDF

    http://bayfiles.net/file/1hkOU/ZmvmWm/The_Fundamentals_of_Perfect_Baking.pdf

    for myself, its based upon this video here

    [ame="http://www.youtube.com/watch?v=kGszEIT4Kww"]The Fundamentals of Perfect Baking - YouTube[/ame]

    just some compilation for learning purposes. I take no credit for the content.

    what do you guys think of it? should i add something more?
  • final_fight
    Offline / Send Message
    final_fight polycounter lvl 9
    It's ok to use 16-bit half float for normal maps?
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    It depends what you mean by use really.
    In a game the best you can hope for is BC5 on current gen, which is a high quality 8bit format.
    I would always recommend baking to 16bit (float or integer) when authoring textures. :)
1679111215
Sign In or Register to comment.