Home Technical Talk

Making sense of hard edges, uvs, normal maps and vertex counts

Replies

  • Neox
    Offline / Send Message
    Neox godlike master sticky
    DKK wrote: »
    My headache:

    Rendered in: Maya Viewport, Kodde PBR shader.
    Baked in: Xnormal, Exported Normals
    Hard edges: At all UV seams.

    Displays very nicely! I like it a lot! However, there are these small issues in blue, No matter what I try they will not go away, I have done multiple bakes, viewed in multiple engines, and this is by far the nicest looking results I have gotten.

    PcountSup.png

    Issue one: a hard looking seam at the end caps, visible on both ends.

    issue two, visible edge hardness along a hard edge(not visible anywehere else on cylindrical shape.

    Very hard to see these issues in sketchfab, but they are there.
    [SKETCHFAB]5b9bdbab145043959ebfbc6e5267b6ac[/SKETCHFAB]

    xnormal isn't synced with maya, so bake an objectspace map and use handplane to convert it to maya tangentspace
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    .Wiki wrote: »
    You don´t need hard edges at all uv seams. You need a UV seam where a hard edge is ;).

    Basically for the sword shed you won´t need any hard edges because its a very smooth object. If you experience some weird lowpoly shading just add 1 or 2 edges.
    Sure, you don't need to have hard edges on all UV seams but it still makes sense to add them because it's free and it makes a less colourful normal map which helps with a) texture compression, b) workflows where the engine and baker are not 100% synced and c) situations where texture resolution isn't great enough to sufficiently represent the curvature of a surface.
    I would even argue that there is no compelling reason not to have hard edges at all UV borders, even when the mesh itself isn't hard ;)
  • SuperFranky
    Offline / Send Message
    SuperFranky polycounter lvl 10
    I would even argue that there is no compelling reason not to have hard edges at all UV borders, even when the mesh itself isn't hard ;)

    There's actually a few reasons not to do that. With mirrored meshes where you have 2 island representing the mirrored surface it may not be a good idea to split these two island to different smoothing groups, especially if your mesh isn't a flat plane. Also, it's not a very good idea to make hard edges on a cylindrical or a curved mesh that was cut up to several pieces in UVs. Situations like that aren't very common, tho, so you should always start with making every uv island a separate smoothing group and go from there after a test bake.
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    There's actually a few reasons not to do that. With mirrored meshes where you have 2 island representing the mirrored surface it may not be a good idea to split these two island to different smoothing groups, especially if your mesh isn't a flat plane. Also, it's not a very good idea to make hard edges on a cylindrical or a curved mesh that was cut up to several pieces in UVs. Situations like that aren't very common, tho, so you should always start with making every uv island a separate smoothing group and go from there after a test bake.
    Thanks for the reply. You say that these situations are not a good idea but you don't say why this is so. Could you clarify please?
  • SuperFranky
    Offline / Send Message
    SuperFranky polycounter lvl 10
    Thanks for the reply. You say that these situations are not a good idea but you don't say why this is so. Could you clarify please?

    I don't know exactly how it works, so my guess is as good as yours. Maybe someone else can give an explanation. But I do know that it's very likely to get a visible seam on mirrored objects in that situation. Just something to keep in mind in case you encounter a seam and you don't understand why it's there. I never did any real tests on this, tho, so take it with a grain of salt.
  • Farfarer
    The only reason I think it'd happen is if the triangulation was calculated differently on either side of the smoothing split. That could easily wind up with the normals and/or the tangents&bitangents being calculated differently - and that would throw off the normal map.
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    I don't know exactly how it works, so my guess is as good as yours. Maybe someone else can give an explanation. But I do know that it's very likely to get a visible seam on mirrored objects in that situation. Just something to keep in mind in case you encounter a seam and you don't understand why it's there. I never did any real tests on this, tho, so take it with a grain of salt.

    Thanks for the reply.
    I'm still going to have to disagree with you on these points.The only time you should ever get seams with mirrored objects is if you have made an error with the geo (as Farfarer said), the baker and renderer aren't synced or the sign isn't stored.

    With a perfectly synced workflow you should be able to have every face as a separate smoothing group, mirrored or not, and still have it render perfectly and without any noticeable seams on flat or curved surfaces.
  • SuperFranky
    Offline / Send Message
    SuperFranky polycounter lvl 10
    Oh, I think you're right. I forgot about the fact that the triangulation plays the role here. Thanks for correcting me.
  • Neox
    Offline / Send Message
    Neox godlike master sticky
    xnormals edge padding is also just a post effect so there should be no isses.
    but what are we seeing in image 1? in 3 it looks like what i would expect besides some bumpyness very slight issues which could be part of the highpoly.

    either way, what will be your target anyways? just a presentation in maya?
  • final_fight
    Offline / Send Message
    final_fight polycounter lvl 10
    The problem I get with this "hard edges" workflow is that there isn't enough curvature on edges. Because of this Quixel Suite fails to create suitable curvature map and masks from it.
  • EarthQuake
    What? Bake a proper curvature map from your highpoly, your lowpoly topology should not influence it in any way.
  • final_fight
    Offline / Send Message
    final_fight polycounter lvl 10
    EarthQuake But DDO creates its own curvature map from input normal map. With averaged low poly mesh everything's fine, but with hard-edged at UV seams...
  • Joopson
    Offline / Send Message
    Joopson quad damage
    final_fight, if the way DDO does it is giving you errors, like EarthQuake says, you can bake your curvature map elsewhere, and DDO will use that instead of generating one itself.
  • final_fight
    Offline / Send Message
    final_fight polycounter lvl 10
    Joopson I know Xnormal can generate curv map. I just want to know - does anybody else get these kind of problems in DDO with "hard-edged" normal maps?
  • EarthQuake
    Generally no, I get more problems if my mesh has no hard edges, as then the smoothing compensation in the normal map is baked into the curvature map.
  • final_fight
    Offline / Send Message
    final_fight polycounter lvl 10
    EarthQuake Do you use DDO? Do you create your normal maps with hard edges? Maybe I do something wrong... First I unwrap low poly, then assign smoothing groups with TexTools, after that I bake normal maps in Xnormals - do I miss something?
  • Bek
    Offline / Send Message
    Bek interpolator
    does anybody else get these kind of problems in DDO with "hard-edged" normal maps?
    No. But you haven't shown exactly what the resulting curvature problem is. I'd suggest posting in the quixel sub-forum with the highpoly / lowpoly / baked normal map and whatever the problem in DDO is. Or just make curvature externally (which should be easier when quixel releases their free GPU baker)
  • AdvisableRobin
    Offline / Send Message
    AdvisableRobin polycounter lvl 10
    Joopson I know Xnormal can generate curv map. I just want to know - does anybody else get these kind of problems in DDO with "hard-edged" normal maps?

    You should post an image of the problems that you get.
  • Yury
    Offline / Send Message
    Yury polycounter lvl 8
    final_fight
    You might also need to check the edges on your high poly model. If they are too tight and take only like 1-2 pixels in your normal map after the bake, this could be the reason why dDo can't generate a proper curvature map.
    I use Quixel Suite a lot for my current hard surface project and I never had this problem
  • GreenShade
    Guys, I have a question for Blender users - or anyone else who can explain me what I'm doing wrong.
    When I try to make a tangent normal bake, I'm always getting an error message that says "feedback loop detected". It means that there is something wrong with either the highpoly or the lowpoly mesh right?
    The strange thing is that I can bake literally everything else: AO, shadows, full render, even object and world based normal maps. So I don't know what could be the problem.
    Please help!
  • Nosslak
    Offline / Send Message
    Nosslak polycounter lvl 12
    GreenShade wrote: »
    Guys, I have a question for Blender users - or anyone else who can explain me what I'm doing wrong.
    When I try to make a tangent normal bake, I'm always getting an error message that says "feedback loop detected". It means that there is something wrong with either the highpoly or the lowpoly mesh right?
    The strange thing is that I can bake literally everything else: AO, shadows, full render, even object and world based normal maps. So I don't know what could be the problem.
    Please help!
    Are you using the same material across both the highpoly and the lowpoly mesh (with the normalmap you're baking to active)? This has caused that problem for me in the past with a similar message. It occurs bacause Blender then needs to both read and write to the texture at the same time, which results in a feedback loop. The solution is to just assign a different material to your highpoly models or tick off the normalmap (for the material) during the baking.

    Personally I prefer xNormal or Substance Designer for baking as you can get some more useful maps out of them with less work IMO.
  • GreenShade
    I had more separate objects each with different material for highpoly, and a single object with it's own material for lowpoly. The highpoly isn't even unwrapped because I only need it's normals but I don't think that's the problem. I tried to merge the highpoly parts, I tried to use the same material, I even tried it without material, I tried to move the parts to a different layer to bake them separately but nothing helps.
    My last idea is to break the lowpoly to it's UV groups and try to bake them one by one then put them together in PS (I know it's dumb but I run out of ideas)

    EDIT: baking the UV groups one by one doesn't work either, it gives me the error message every time, for each group - looks like I will have to download one more software after all...
  • ekul
    Just a quick question, are triangle fans common/suitable for a low poly object in terms of a proper bake and in-game rendering?

    triangle-fan.png
  • Farfarer
    Generally you'd want to avoid fans like that if possible. They can cause lighting issues and they aren't the most efficient capping method for cylinder shapes in terms of both rendering and polycount.

    You could delete that fan then bridge the side with the forks over to the opposite side and get a bit of a cleaner mesh.

    Although in this specific case, it's not creating a lot of very long or thin triangles converging on a single vertex. That's when triangle fans can become really problematic. So you're probably fine to leave it.
  • cataev serj
    Please look at my example below:
    hZfSejB.jpg

    Baked it in substance designer. Followed the rule of thumb about uv seams and smoothing groups. Maybe the lowpoly mesh has bad topology?
  • teho
    Offline / Send Message
    teho polycounter lvl 7
    did you baked with a cage?
  • cataev serj
    Nope. I'll try now and show the results.
  • cataev serj
    I see the difference. Thanks for tip.
    iB8YomA.jpg
  • teho
    Offline / Send Message
    teho polycounter lvl 7
    no prob, always use a cage, it's a timesaver because you can bake any map without artifacts, espacially if there is a hard edge and diff smooth groups.
  • teho
    Offline / Send Message
    teho polycounter lvl 7
    hi everyone, i have an odd problem,
    the .obj mesh is ok, but .fbx mesh has hard edge...
    any ideas why? i'm previewing in marmoset.
    RJYgFpO.jpg

    thanks.
  • AdvisableRobin
    Offline / Send Message
    AdvisableRobin polycounter lvl 10
    Check your export options, probably one of them is exporting smoothing groups and the other isn't.
  • teho
    Offline / Send Message
    teho polycounter lvl 7
    both with smoothing groups.. and there is a strange thing, because on the opposite of fbx model no seams and normal map rendering correct. i thought maybe symmetry modifier inverted for other side normal, but it didn't. i checked that fbx at Unreal - no problems, so it is only in toolbag...
  • tonyd927
    didnt wanna make a totally new thread for this at is a pretty noobish question, but how do I align the rotation of a UV island in Maya? Similar to Textool's Align rotation to edge tool for 3Ds Max. It's been bugging me that I cant figure this out. Thanks!
  • huffer
    Offline / Send Message
    huffer interpolator
    Teho, are you exporting fbx with tangents and binormals? It might be a tangent space issue or flipped green channel.
  • Dklang
    i have a couple of noob questions. Im trying to figure out the simplest workflow. I just started using Quixel tools, would be great if there is a tutorial about them. And i think i got this metal crate from another thread: http://i.imgur.com/cYKglOl.jpg Its pretty high poly, if I break it down i get confused about what part of it is modeled and what part is a normal map? Or are these screws for instance sculpted there? Like are these objects made individually and combined together to form a box? And is such high poly models smooth previewed or using actual smooth?
  • samholy
    thank you so much. i learned a lot of basic theory i never saw anywhere else.
    helped me understanding many issues i had with a character
  • Quack!
    Offline / Send Message
    Quack! polycounter lvl 17
    This is a normal map synching problem.

    If you are working in an old school un-synched workflow, like displaying in the max viewport, then you will need to work hacky and use extra smoothing groups at hard edges and split uv islands to lessen the normal map gradation. This will lessen those errors.

    Using the 3point Quality Normals simulates a synched workflow.

    Xnormal Mikkt Space
    Substance Mikkt Space
    Knald

    The above all sync up with modern engines like unity, ue4, toolbag2.

    To sync up I would bake in xnormal or substance with a cage and select the appropriate tangent basis inside of TB2 and it should look better.

    Also you want to make sure you have at least 16px of padding/dilation on your uv islands in the normal map. In fact you can never have to much.
  • EarthQuake
    If you're baking in max, I would suggest previewing in Marmoset Toolbag 2 and setting the tangent space to max. Recent versions of Max have broken accurate tangent space display iirc.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    Did you change the normal map setting in Marmoset to Mikk (I think that's what the xNormal one is named)?
  • EarthQuake
    Do that, and make sure to triangulate before baking. You can change the tangent space in the mesh's properties.
  • hostis
    Hello, everyone! Think I'd better ask it here, then open a new tread. So I'm trying to make my first bake, nothing special for the beginning, just a cube, but I'm already stuck and puzzled with it. Sorry, for newbie questions in advance.-) So I have a cube and bake normals in tangent space. As far as I understand, they depend on the local object space of the cube. So if I turn my cube upside down and bake it the green channel will be at the bottom(in the world space), because the local Y coordinate of the cube will point downward. In other words, it doesn't matter what the cube orientation in the world space is the "up" for the normal map is where the local Y axis points. And got this:
    norm.png]

    Ok, it even looked more or less good in the 3DO window.
    norm_in_ddo.png

    I, actually, decieded to learn to bake normal maps for texturing in DDO in the first place.

    And then I added my first smart material and it didn't work the way it should.

    gun_mat.png

    On the left is how material looks in 3DO and on the right spec map derived from the normal map shown above. So, for unknown to me reason, DDO only weathers edges which point up and right in the normal map and doesn't see the others. I asked this question in the quixel subforum, but couldn't find a way around. So I continued my cube tests and found another thing that puzzled me. No matter how I rotate the cube uvs they always look the same in the uv window: the "up" is always green and the "right" is always red, even if it doesn't correspond to the cube's "up" and "right". Then I turned one uv shell 45 degrees and was puzzled even more. It was like this:
    box_turned_03_normals.png
    Basically, all the four uv sides changed colors.

    So, my questions:

    1)Why the uvs doesn't change their colors if I rotate them in the uv editor window? If I turn them so that what's in local object space's "up" points right in the uv window, should't the green appear on the right during the bake?

    2) Can anyone say what's wrong with my normal map for DDO, and why DDO sees only edges that point upwards and rightwards and how to fix it? That's the most intresting question.

    Sorry, again for such a long post on such basic questions.
  • Farfarer
    Tangent space normal maps like that record the direction the surface points in texture space, not local/object space.

    What this means is that red will always point towards the right of the texture and green will always point towards the top (or the bottom, depending on whether you use an inverted green channel or not).

    The orientation of the UV islands doesn't matter - red and green are always left/right and up/down when you look at the normal map itself.

    It's what makes tangent space maps so flexible, especially for tiling normal maps - you can apply them to arbitrary meshes and they'll always work because it's stored in texture space, not the object's space.
  • hostis
    Thanks, Farfarer! One question less.-)
  • xChris
    Offline / Send Message
    xChris polycounter lvl 10
  • Thane-
    Offline / Send Message
    Thane- polycounter lvl 3
    Can someone illustrate an example of a good reason or result of using a smoothing group in addition to the averaged normal cage in xnormal? I don't quite get that part or why its of value. Is it so large faces are easier to paint on in photoshop or apply details, ie they are more just the normal blue color?

    I thought Xnormal calculated normals differently than say Max, resulting in the normal map showing wrong on the model whos normals were calculated in Max.
  • dirigible
    Offline / Send Message
    dirigible polycounter lvl 8
    Thane:

    When you average normals, each face affects the shading of the other faces it touches. So the normal map for each face is changed to account for all the other faces near it - even if they're separated by a UV seam.

    The problem is, when you have a UV seam sometimes the pixels don't perfectly match up with the UV seam. You get pixel bleeding that causes the edges to not quite match up, when you apply the texture.

    So what you can do is harden the normals at the UV seams. This means that those faces' shading won't be affected by each other, and the normal map won't have to account for that. As a result, the normal map will be less intense at the seams, and so there will be less of a chance of seeing any potential seams.

    HOWEVER, when you're baking using a cage, the cage "looks at" your model using its normals. If you have a split in your cage's normals, then you'll have a split in the cage's "vision". Your cage won't be able to see part of your model - or it might potentially see the same part of your model twice. So you want your cage to have all averaged normals.

    IMPORTANT NOTE:
    This technique works only for static meshes. If you're going to rig and deform your mesh (like if its a character) then you should not harden your UV seams. It will look like its helping, up until your model starts deforming, and then you'll get huge visible seams.
  • Thane-
    Offline / Send Message
    Thane- polycounter lvl 3
    So a correct workflow would mean i make the low poly in Max and apply smoothing groups where UV splits are or where i need them (but i must create a UV split with padding if i smooth somewhere there is not already a UV split), then in Xnormal under the low poly settings choose "average normals" and also load in a cage and ignore everything else.

    The the cage itself need its normals averaged normals for this technique, like does it need one smoothing group? If it does im going burst out laughing and never stop.
  • EarthQuake
    I think you should try experimenting with this a bit, you seem to be getting wrapped up in technical information, much of this should be apparent if you try.

    Some more info:
    1. average normals in xnormal would override your mesh normals, you should not do this
    2. you can use hard edges wherever you have uv seams, you don't need to, but you can, it doesn't add to the vertex count and can lessen the intensity of gradients in your normal map (the first post explains this in great detail).
    3. wherever you have hard edges, you need to have a uv split with padding, otherwise you will get artifacts at the seams
    4. if you use hard edges, you need to make sure you're using a cage (aka averaged projection mesh), otherwise you will get artifacts (gaps) at the seams. You don't need to do anything special to average your cage, if you use a cage at all it will be averaged. Not using a cage would be using the offset method in max, or using the default ray distance settings in xnormal. For Max, the cage is on by default, for XN, you need to make a cage with XN, or import one. For maya, you need to set match using to geometry normals rather than surface normals (I think, I sometimes get these two confused).
  • EarthQuake
    dirigible wrote: »
    Thane:

    IMPORTANT NOTE:
    This technique works only for static meshes. If you're going to rig and deform your mesh (like if its a character) then you should not harden your UV seams. It will look like its helping, up until your model starts deforming, and then you'll get huge visible seams.

    This is interesting, do you have some examples you can show that demonstrate this?
  • Scizz
    Offline / Send Message
    Scizz polycounter lvl 11
    Alright, I'm stumped. This hasn't been an issue before. Where ever there is a hard edge, you need a UV split. And where ever there is a UV split, you need a smoothing split, right?
    3f99939729.png
    Hi-Poly
    u6JXQYX.png
    UVs and their smoothing groups
    d9D4V55.png
    The mesh in Marmoset with the xnormal baked, normals.
    726d5cf6fb.png
    My normal map.
    It's as if the normal map isn't even applied
Sign In or Register to comment.