Home Technical Talk

Making sense of hard edges, uvs, normal maps and vertex counts

1568101115

Replies

  • anaho
    Offline / Send Message
    anaho polycounter lvl 8
    I am still getting seams with caged baking exported from Blender 2.69 RC to XNormal.
    1.) UVs are set up correctly with a lot of padding space, hard seams are in place
    2.) Lowpoly serves as slightly scaled up cage
    3.) Lowpoly and cage have only ticked "Selection Only", "Include UVs" and "Smoothing Groups"
    4.) When importing .obj back to Blender seams are visible along the bake, where the seams where...

    There are a lot of normal options in Xnormal as well. Maybe one of these is the culprit?
    In theory with the new smoothing group option it should be pretty easy to get a caged bake just from Blender and Xnormal ...
  • Nosslak
    Offline / Send Message
    Nosslak polycounter lvl 12
    You should post an image of what your model looks like as it'd make it a lot easier to detect the problem.

    I haven't tried the new smoothing groups options for Blender and still just use edgesplit so I don't know how well that works. So you could try edgesplit and see if that makes any difference.

    Otherwise you can try baking OS normalmap and bringing that into handplane.
  • anaho
    Offline / Send Message
    anaho polycounter lvl 8
    It is exactly the same problem as posted one page before by NanoTurtle.
    Applying the edgesplit must not be done, unless you want double verts and that messes up your mesh essentially. I mean thats essentially the whole point why the new "Smooth Groups" option is there after all.
  • Nosslak
    Offline / Send Message
    Nosslak polycounter lvl 12
    Yeah, I know why you'd want to use the smoothing groups option, I just thought you could see if it makes any difference with the edgesplit. You should get the same results while baking with or without the new smoothinggroups option as long as you use a cage.

    I think you'll need to include the normals for the smoothing-groups to work. I haven't used them but it feels like that might be your problem. You should also make sure to triangulate the model upon export so xNormal doesn't triangulate it differently.
  • EarthQuake
    Nosslak wrote: »
    You should get the same results while baking with or without the new smoothinggroups option as long as you use a cage.

    Not if the edges are physically split, you'll get gaps.
  • mAlkAv!An
    Offline / Send Message
    mAlkAv!An polycounter lvl 5
    Make sure to use 'Bitflag Smoothing Groups', this is what works fine other apps as well.
    You can also uncheck smg and just 'Include Normals' which will export hard edges via split vertex normals.
  • Nosslak
    Offline / Send Message
    Nosslak polycounter lvl 12
    EarthQuake wrote: »
    Not if the edges are physically split, you'll get gaps.
    I'm sorry, maybe I just weren't clear enough. What I meant was that if I physically split the edges or use smoothing groups together with a cage (for both the physically split and smoothing group model) the result will be the same. In this scenario the cage was created before the edgesplit modifier is applied (which splits the geometry upon export) so the cage you create is scaled up along the normals of the model before the split (i.e with the averaged normals).

    I know that if I were to just physically split the model and then create a cage from that I would get gaps.
  • anaho
    Offline / Send Message
    anaho polycounter lvl 8
    Well, tried it with only the bitflag and include normals options, seams still prevail unfortunately. Maybe I just set up the cage wrong or it is an option in Xnormal but seriously how hard can it be to get correct bakes on such simple object. If someone wants to try here is the http://www.pasteall.org/blend/24886
  • Defaultsound
    Offline / Send Message
    Defaultsound polycounter lvl 14
    I don't know if anyone could shed some light on this but I've come across a conflict in information.

    In regards to normals maps I've always been told that you can edit them in Photoshop fine and overlay or tweaks things that don't bake out correctly.
    Don't be afraid to edit normal maps in Photoshop. After all it is just a texture, so you can clone, blur, copy, blend all you want...

    Quoting the Wiki it also says that this is fine.

    Then I watched this hour long video on Normal maps and now I am confused.

    [ame="http://www.youtube.com/watch?v=OONQzKcWeMY&feature=share&t=27m59s"]Andy Davies - Normal mapping for games & realtime content - YouTube[/ame]

    28 minutes in he goes on to saying that you shouldn't edit normal maps in Photoshop, since they aren't 2D textures. Editing them forms bad practice and messes with things behind the scenes.

    Is this entirely true?

    I couldn't get my head around it. Either way I've decided that in some cases editing is often a necessity.
  • Farfarer
    Generally, it's not a great idea. It's a normalised set of vectors that - if it's not a tiling texture - are specifically synced to a tangent basis. Photoshop has no real way to understand that.

    That said, if the normal map has come from something like crazybump from a photo, then they're already inaccurate enough that editing them in Photoshop will make little difference. So long as you renormalise it afterwards.
  • mAlkAv!An
    Offline / Send Message
    mAlkAv!An polycounter lvl 5
    The talk is referring to high poly -> low poly bakes and therefore it's true that fixing faulty normal maps in photoshop due to poor bakes is bad practice. Something which has been stated in multiple polycount threads several times.

    anaho wrote: »
    Well, tried it with only the bitflag and include normals options, seams still prevail unfortunately. Maybe I just set up the cage wrong or it is an option in Xnormal but seriously how hard can it be to get correct bakes on such simple object. If someone wants to try here is the http://www.pasteall.org/blend/24886

    Just gave it a try and it looks fine here. Did you by any chance just forgot to add an Edge Split modifier for actually previewing the hard edges?
  • Defaultsound
    Offline / Send Message
    Defaultsound polycounter lvl 14
    Okay cheers guys. Clears things up, seems like I've learnt more and more to normal maps in the past year, for example UV splits was another thing I never knew about. Hopefully there won't be any more surprises.
  • anaho
    Offline / Send Message
    anaho polycounter lvl 8
    @ mAlkAv!An Thanks, that was exactly how it looked for me. But if you look really closely you can see the edges running around especially on the extruded box. It becomes more visible if the diffuse color is not white IMO. You really have to look twice to see it, maybe this is just how it supposed to look after all and I am getting to anal about this.
  • Noors
    Offline / Send Message
    Noors greentooth
    Nice vid. He summed up pretty much what has been discussed here many times.
  • mAlkAv!An
    Offline / Send Message
    mAlkAv!An polycounter lvl 5
    @anaho
    These seams are caused by low resolution or lack of anti-aliasing. Not much you can do about if you need to keep your textures at a reasonable size. Baking at 4x resolution and then scale down afterwards might help a little (so that it's actually more than 4xAA).
  • CognizanCe
    Offline / Send Message
    CognizanCe polycounter lvl 10
    Ty so much for the info.

    I used to do all my baking using the method A in the first post.
    Of course, hard surfaces came out fubar.

    Right now, I'm using the " C: Hard edges at uv seams, "Explicit mesh normals" projection" and my results are much better, but I cannot use the B: Hard edges at uv seams, "Averaged projection mesh"method because I cannot get cages to work.

    I'm trying to use the cage method mentioned in this vid:
    [ame="http://www.youtube.com/watch?v=kGszEIT4Kww"]The Fundamentals of Perfect Baking - YouTube[/ame]
    jump to 15:18
    My maps get messed up when I try this for the "Averaged projection mesh"method.


    Also,
    In your pic below
    mnt_01.jpg

    How would you create a cage for an object like that?
    Scaling uniformly from the center pivot would not encompass the entire high poly mesh, so would you move verticies around by hand, since it's low poly and wouldn't be too hard to do?
  • .Wiki
    Offline / Send Message
    .Wiki polycounter lvl 8
    CognizanCe wrote: »

    How would you create a cage for an object like that?
    Scaling uniformly from the center pivot would not encompass the entire high poly mesh, so would you move verticies around by hand, since it's low poly and wouldn't be too hard to do?
    Just push the vertices out. Theres no need to move every vertice along its normal. Just do it with a simple "push" operator.
  • CognizanCe
    Offline / Send Message
    CognizanCe polycounter lvl 10
    .Wiki wrote: »
    Just push the vertices out. Theres no need to move every vertice along its normal. Just do it with a simple "push" operator.

    I guess you're a max user.

    I think the video described the correct process for maya, because vahl describes it here:

    http://www.polycount.com/forum/showthread.php?t=38643
  • .Wiki
    Offline / Send Message
    .Wiki polycounter lvl 8
    CognizanCe wrote: »
    I guess you're a max user.
    Nope, Softimage but the process of building the cage is the same in all applications.

    The push operator does nothing more then moving all the vertices along their corresponding surface normals, so selecting all of them and moving them along their normal does the same like a push operator.
  • CognizanCe
    Offline / Send Message
    CognizanCe polycounter lvl 10
    .Wiki wrote: »
    selecting all of them and moving them along their normal does the same like a push operator.

    Correct.
    But like I said, my normal bakes get messed up when I use that method.

    It should work/be that simple in theory, but I dunno if I'm doing something wrong.
  • MDiamond
    Offline / Send Message
    MDiamond polycounter lvl 10
    I believe I'm having the same issue as CognizanCe. I don't seem to get my cages to work.

    I'm having problems baking a normal map of this simple cube.
    I'm using Modo 601 and xNormals. The Low Poly has separate UVs and Smoothing Groups accordingly, and a cage created in Modo.

    However, after the bake I get the weird "puffed out" effect on the edges. From what I got so far this seems to be a problem that happens when you don't use a cage, but I am certainly loading my cage when using xNormals, even checking in the 3D viewer. I'm using xNormals default settings, so I'm not sure if there's something else I'm missing.
  • Joopson
    Online / Send Message
    Joopson quad damage
    Are you softening all the cage's normals?
  • MDiamond
    Offline / Send Message
    MDiamond polycounter lvl 10
    Not sure. Should I? I literally just clone the Low Poly mesh(with UVs, Smoothing Groups and all the junk that may be cloned in the process) and use Push Tools/Move vertexes along normals to increase the cage's volume a tad. I thought only the cage's vertex information were important in the baking process so I never bothered to assign/clear Smoothing Groups in the cage.
  • Joopson
    Online / Send Message
    Joopson quad damage
    MDiamond wrote: »
    Not sure. Should I? I literally just clone the Low Poly mesh(with UVs, Smoothing Groups and all the junk that may be cloned in the process) and use Push Tools/Move vertexes along normals to increase the cage's volume a tad. I thought only the cage's vertex information were important in the baking process so I never bothered to assign/clear Smoothing Groups in the cage.

    Yeah, you have to make sure the cage's normals are soft. My guess is that'll fix your issue.
  • EarthQuake
    MDiamond wrote: »
    Not sure. Should I? I literally just clone the Low Poly mesh(with UVs, Smoothing Groups and all the junk that may be cloned in the process) and use Push Tools/Move vertexes along normals to increase the cage's volume a tad. I thought only the cage's vertex information were important in the baking process so I never bothered to assign/clear Smoothing Groups in the cage.

    How are you importing the mesh into xnormal?

    Are you importing the "cage" that you created directly into the lowpoly slot?

    You'll need to load the regular lowpoly into the lowpoly slot, and then I beleive there is a special tab in the lowpoly mesh options to load a corresponding cage mesh. "External cage file" - load it in there.
  • MDiamond
    Offline / Send Message
    MDiamond polycounter lvl 10
    Came back from work and made a quick test in Maya. Everything went just fine. I tried with the cage with completely soft edges and another with hard edges and both yielded the same correct results. I use Modo 601 at work and this last test made me think that maybe the issue is within Modo. Perhaps I should try another export format. I'll make a quick test in my own Modo 701 and post the results.

    I appreciate the support, and looking forward to share more with the community :)
  • Bek
    Offline / Send Message
    Bek interpolator
    If you're using modo you'd be insane not to use PipelineIO for this.

    If you think the exports from modo are causing trouble; check in xNormal's viewport to verify that the low/cage are correct. If you want to export stuff manually you can use Fararer's vertex normal toolkit to set soft/hard edges manually. Basically to manually do what pipelineIO does automatically, triangulate your low, set hard edges, export. Use push, soften all edges, export cage.
  • MDiamond
    Offline / Send Message
    MDiamond polycounter lvl 10
    If you're using modo you'd be insane not to use PipelineIO for this.

    If you think the exports from modo are causing trouble; check in xNormal's viewport to verify that the low/cage are correct. If you want to export stuff manually you can use Fararer's vertex normal toolkit to set soft/hard edges manually. Basically to manually do what pipelineIO does manually, triangulate your low, set hard edges, export. Use push, soften all edges, export cage.

    Thanks, will definetively check it out.
  • theGrandPoobah
    Offline / Send Message
    theGrandPoobah polycounter lvl 8
    I am a little late to this thread, so apologies.

    I am in the middle of reading a book about game textures (pub'd 2011) and in it the author mentions that texturing is different from skinning and that his book doesn't go into applying skins to 3d meshes.

    So let me ask this: is there any recommend literature or video tutorials on the subject of skinning complex 3d model/meshes?


    Preferably free material, but if there's a strong recommendation, I'll check it out!
  • MDiamond
    Offline / Send Message
    MDiamond polycounter lvl 10
    So, I imported the test I did in Maya previously and applied the map generated in xNormals in Modo at work and I still got the issue. I then tried to view the exact same model with the same map in Marmoset and it is showing the model just fine. So I guess this is some sort of display issue in Modo's viewport.

    @Bek
    Trying the PipelineIO scripts, really time saving plugin. Enjoying the customizable options.
  • Bek
    Offline / Send Message
    Bek interpolator
    Oh, whoops—set your Normal Map opacity to 50% in Modo.

    No idea why 50% is correct but it just is. You can see an example here, section 6f.
  • Farfarer
    Yeah, it's a rendering bug in 601. It's fixed in 701.
  • CognizanCe
    Offline / Send Message
    CognizanCe polycounter lvl 10
    Joopson wrote: »
    Are you softening all the cage's normals?

    I think I agree with joopson, your high poly mesh looks rather soft.
    Having hard edges on a low poly from a soft high poly mesh won't looks right (the way it does on your foreground cube)
  • MDiamond
    Offline / Send Message
    MDiamond polycounter lvl 10
    Ok, so I guess that was the source of the trouble. Glad to know its fixed in newer versions of Modo.
  • Farfarer
    The texture itself will be fine, by the way - it's just that the viewport shows it incorrectly.
  • FourtyNights
    Offline / Send Message
    FourtyNights polycounter
    I baked a normal map for my hard surface model and I got pretty good results without major problems. I'm somewhat happy with my results.

    I triangulated my LP and CAGE models before baking them in xNormal. When I applied the normal map for my original quad version of the model just for the sake of testing, some odd shadings appeared. In the case of hard surface model I don't care much, since those normals looked great on the triangulated version.

    I was wondering that, how the baking is to work with organic models, such as human beings? Because of rigging and animation, organic models need 100% quad mesh for good deformations. And I guess rigging with textures applied it's important to see how joints deform, etc.
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    I baked a normal map for my hard surface model and I got pretty good results without major problems. I'm somewhat happy with my results.

    I triangulated my LP and CAGE models before baking them in xNormal. When I applied the normal map for my original quad version of the model just for the sake of testing, some odd shadings appeared. In the case of hard surface model I don't care much, since those normals looked great on the triangulated version.


    I'm not surprised that it looks wrong tbh. You are essentially using two different meshes as far as the normal map is concerned. ;)
    You always need to use exactly the same triangulated topology on your game ready mesh as the mesh you used to bake with or it will break the normal map. The reason you are seeing the errors in the quad mesh is that the mesh is being triangulated under the hood in such a way that it doesn't match your baked topology.
    Quads are not really quads in the same way that ngons are not really ngons. Everything is triangulated at the most basic level when rendered and then made to look like quads or ngons in the viewport for the artists convenience simply because its easier to look at and work with that way.
    Each quad that you have has a 50% chance of being flipped in the opposite direction than your bake which is why it is important that you use the same topology as often internal triangulation isn't something you have control over.
    I was wondering that, how the baking is to work with organic models, such as human beings? Because of rigging and animation, organic models need 100% quad mesh for good deformations. And I guess rigging with textures applied it's important to see how joints deform, etc.

    I'm not sure where you got this information but it is wrong. :(
    You can get perfectly acceptable results with a triangulated mesh and often it can actually help when a mesh is triangulated.

    LimbTopology?action=AttachFile&do=get&target=BenMathis_limb_deformations.gif

    http://wiki.polycount.com/LimbTopology?highlight=%28\bCategoryTopology\b%29

    BodyTopology?action=AttachFile&do=get&target=topo_StreetCop1_MashruMishu.jpg

    BodyTopology?action=AttachFile&do=get&target=topo_Bayonetta01_wire.jpg

    Either way all meshes are triangulated in your game engine when rendered so all the quads are gone. at that stage.
    In addition to this, until recently it hasn't been practical to use all quad meshes even if they could be rendered as quads in game because it costs more in terms of geometry which has been at a premium when memory has been so tight. As such artists have been forced to triangulate their meshes in order to optimise and stay under budget.


    TL;DR: For all game ready assets you must always use exactly the same triangulated topology on your game ready mesh as the mesh you used to bake with or you risk breaking the normal map. No ifs or buts. :)
  • JohnBarnard
    Hello people of Polycount!
    I've been doing a lot of normal map baking recently, and this thread has improved my knowledge of the subject immensely.Thank you all so much for your contributions!

    I have a quick question/example that I wanted to get your opinions on...
    Here are a couple of windows I'm making for an exterior environment:

    yEi4yoF.jpg?1


    My question: Would the end result in this example be considered an acceptable baking solution?

    Whilst I *think* I've got a good understanding of normal map baking, assets like these windows sometimes keep me guessing, because all those UV splits and supporting edge loops seem pretty expensive!

    Generally, my workflow is:
    1) Hard edges at UV seams
    2) Supporting edge loops where required
    3) Bake using averaged projection normals

    That is exactly how I baked these windows. They look just as I wanted, but I'm concerned that the large number of UV shells and supporting edge loops look amateurish.

    Am I doing this right? Is there something else I should be considering?


    Any opinions are welcome.

    Thanks!
  • snakeLK
    sign!!!!!!!!!!!!!!!!!!!!!!!!!!
  • Damian Nachman
    Offline / Send Message
    Damian Nachman polycounter lvl 6
    So... I've been wondering.
    When baking hard surfaced models, I can either put UV seams on hard edges, or just bevel those edges instead. (Which will result in more geo).
    But when I'm dealing with low poly assets, sometimes I just cant afford all those bevels, So I end up with a crazy amount of UV seams, which render my model untexture-able.
    In this case, is stitching up all the uv shells my only solution?
    (If so, then I guess I should just create models that contain less hard edges).
  • ZacD
    Online / Send Message
    ZacD ngon master
    Every UV seam and hard edge counts as an extra edge in game engines. You want to avoid too many UV seams, it makes UVs harder to work with, increases your final vertex count in game engines, and can be a time sink when working with the UV layout and textures.
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    Every UV border in UV space that isn't already a hard edge in 3d space costs extra verts.

    Adding a single bevel costs exactly the same adding a hard edge too, so you can either have bevels or a hard edge for the same cost. In addition you can add hard edges around your UV borders for free anyway so you could actually have both bevels and hard edges for the same cost as just bevels as long as the hard edges are around the UV borders only.
    :)

    TL;DR Add bevels instead of hard edges and then add hard edges around your UV borders for free.
  • EarthQuake
    Not exactly true, sure the vertex cost is the same but the triangle cost is not. The difference is probably pretty small on modern hardware but extra triangles are not free. But otherwise yeah, the basic concept is sound, you can add bevels or hard edges and the cost difference between the two is minimal.
  • Damian Nachman
    Offline / Send Message
    Damian Nachman polycounter lvl 6
    What a fast response! Pretty glad to hear that bevels aren't as bad as I thought.
    Time to bevel!
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    @EQ, Right, but vertex cost trumps triangle cost anyway.
    Ofc, triangle count is going to differ between hard edge and bevelled meshes but any actual increase in render cost (if any) would be negligible because of the amount if cache on modern GPUs makes it possible to store the result of each vertex so it can be reprocessed essentially for free when duplicated as needed.

    Essentially all that really matters when taking into account rendering cost is the amount of vertices that are to be rendered and that the exporting application orders the vertices in a GPU friendly way, which all exporters should do anyway. If people take into account the vertex count primarily and triangle count secondarily then they will in the vast majority of cases over compensate in their favour.
  • Kyalie
    Offline / Send Message
    Kyalie polycounter lvl 12
    Hi guys,
    I'm on maya 2011 at work, and i'm encountering an issue i never had on maya 2009.
    Whenever i triangulate a quad or flip an edge, another polygon of the mesh just has its uvs vanish (it doesn't happen if i triangulate the whole mesh)
    700194uvissue.jpg
    Is anyone getting the same issue? It starting to seriously piss me off...

    (And sorry if that's not the good topic)
  • Boonta
    Offline / Send Message
    Boonta polycounter lvl 13
    hey so when using "Averaged projection mesh" workflow .. (hard edges on uv seams + a cage.)

    in fbx settings should i have any particular options disabled/enabled... notably:

    smoothing groups and binomials and tangents?

    i will be baking in xnormal.
  • Neox
    Offline / Send Message
    Neox veteran polycounter
    Boonta wrote: »
    hey so when using "Averaged projection mesh" workflow .. (hard edges on uv seams + a cage.)

    in fbx settings should i have any particular options disabled/enabled... notably:

    smoothing groups and binomials and tangents?

    i will be baking in xnormal.

    use .sbm then i'd say :)

    it exports your lowoly WITH the prjection modifier, just mae sure you have an dit mes modifier between your editale poly object and your projection modifier
  • Boonta
    Offline / Send Message
    Boonta polycounter lvl 13
    sounds like max speak, im using maya atm... would the history of a transform components function work similarly?

    that aside i'd have to see if you can install the sbm exporter on maya LT... :/....aaannnd it looks like this may not be possible. so ill have to stick with fbx for now...
  • Froyok
    Indeed, you will want to enable your smoothing groups and binormals when exporting from Maya. I would also suggest to stay on the "unweighted" vertex normals mode (you can change this setting in the shape node of your mesh under "mesh controls").

    Also avoid the triangulate setting when exporting in FBX, it's a different algorithm that the one in Maya. You should triangulate manually in Maya to get the proper results.

    Also don't enable "split per-vertex normals" in the FBX settings, otherwise it will breaks your whole shading.
1568101115
Sign In or Register to comment.