Home Technical Talk

Looking for normal map help

So my first ever post to Polycount is a cry for help, or at least for some understanding. This is all new to me and I get very little help from the instructors at my school, so needless to say, I TRY to teach myself everything I can.

I've tried to search for the answer but I don't really know what to ask. So here's my problem. I've created both the hi and low poly models. I'm now to process of generating the normals. In the pic, I pulled the low poly away for the sake of illustration. I'm using XNormal to generate the maps, but I always get, what appears to me, the base mesh showing through. My instructor told me that my "edges are wrong", "your normal angles are wrong", "you need to soften/harden your edges" but I'm never told how to fix/avoid this. Sorry to vent and I appreciate in advance any and all help that I get.

Untitled-2.jpg
forumUntitled-2.jpg

Replies

  • Joshua Stubbles
    Options
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    Your low poly mesh has no smoothing groups on it, it's all faceted. So essentially, all of the normals are broken. Doing that, you're going to have normal bake issues.

    What is the 3D app you're modeling in?
  • ivanzu
    Options
    Offline / Send Message
    ivanzu polycounter lvl 10
    Your low poly mesh has no smoothing groups on it, it's all faceted. So essentially, all of the normals are broken. Doing that, you're going to have normal bake issues.

    What is the 3D app you're modeling in?

    Answer is above,you should always smooth the low poly and high poly one to get good normal map.

    I think that its maya.
  • R6Rider09
    Options
    Offline / Send Message
    Thanks for the quick replies!! It is indeed Maya 2011. I do appreciate the help!!
  • tharle
    Options
    Offline / Send Message
    tharle polycounter lvl 9
    what happens when you normal bake is that an imaginary light ray is cast out, per pixel, from your low poly to your high poly and then the correct angle of light where it hits the high poly is baked down to the texture on the low poly. because you have hard edges all of the light rays of a face are being fired out parallel and you're getting gaps at the edges where no samples are happening. a great way of illustrating this is if you turn off "keep faces together" and then do an extrude on multiple faces. see how theres a gap between the face at the corners? thats exactly whats happening on the bake too - hence the discontinuity in the normals in the bake. if you soften the edges it averages the vertex normals and gives you the smooth bake you're after.

    cheers
  • EarthQuake
    Options
    Offline / Send Message
    tharle wrote: »
    what happens when you normal bake is that an imaginary light ray is cast out, per pixel, from your low poly to your high poly and then the correct angle of light where it hits the high poly is baked down to the texture on the low poly. because you have hard edges all of the light rays of a face are being fired out parallel and you're getting gaps at the edges where no samples are happening. a great way of illustrating this is if you turn off "keep faces together" and then do an extrude on multiple faces. see how theres a gap between the face at the corners? thats exactly whats happening on the bake too - hence the discontinuity in the normals in the bake. if you soften the edges it averages the vertex normals and gives you the smooth bake you're after.

    cheers

    Not always true, this depends on what method you're using to bake, in maya you have "match geometry" and "match normals" or something like that, one of them(don't remember which) works as you say(and should not be used) the other will always be averaged. In max its the same using the cage or the "offset" method.

    But even when your projection mesh is averaged, you need to smooth everything, and where you do have hard edges, you need to break your uvs off. This is because you'll get seam artifacts along that hard edge as both faceted triangles will try to render a harsh line across the same row of pixels.
  • Frankie
    Options
    Offline / Send Message
    Frankie polycounter lvl 19
    As well as the visual artifacts unnecessary split normals can cause it's also worth knowing that as it's essentially duplicating the vertex so it's hugely inefficient when you put it in game.
  • EarthQuake
    Options
    Offline / Send Message
    Well, unless its along a UV border anyway, then there isn't any performance hit as the verts are already doubled there. I always run a script that hardens edges along UV borders, as its good for a lot of other reasons(poorly synced shaders, compression, extracting detain with CB, etc).
  • cptSwing
    Options
    Offline / Send Message
    cptSwing polycounter lvl 11
    ...not when it's the same edges as your uv shell borders, though, from what i gather. mesh gets split here at render time anyway.


    ah bah, ninja'd by an earthquake. there's a japan joke in there somewhere.
  • Frankie
    Options
    Offline / Send Message
    Frankie polycounter lvl 19
    Not really, a uv split would split 1 vert into 2 where as most of them visible on the model could be split between 4 or 8 verts depending on edge direction, there's even a few cases where 10 verts would be possible. Well I think so anyway, I could be wrong...
  • cptSwing
    Options
    Offline / Send Message
    cptSwing polycounter lvl 11
    [FONT=arial,helvetica,sanserif]Another way to look at it in the big picture is to "reuse" vertex splits. For example, I said earlier that renderers allow one material per vertex and one smoothing group per vertex. In other words, if you have a smoothing group and a material ID group that occupy the same set of faces, they'll get split only once. The same goes for UV discontinuities: if they occur at smoothing group boundaries, then they won't cause an extra split to occur. [/FONT]


    *shrug*
  • Frankie
    Options
    Offline / Send Message
    Frankie polycounter lvl 19
    read the rest of the article :)
  • cptSwing
    Options
    Offline / Send Message
    cptSwing polycounter lvl 11
    eh? the ideal case i'm shooting for here is a mesh with one uv channel, one material id, and a normal map baked with hard edges/smoothing groups applied per uv shell. why would adding in several smoothing groups cause more vertex splits than a single smoothing group for the whole mesh seeing as it'll be split along the uv borders anyway?
  • Frankie
    Options
    Offline / Send Message
    Frankie polycounter lvl 19
    In the original poster example every single face is flat shaded, "smoothing group" is a max specific term, without a smoothing group applied everything is flat shaded. When triangles are exported to game with 'flat shading' it is essentially giving each triangle unique vertices. It's the same as breaking them up manually in max but by using smoothing groups or hard edges (or whatever other term a 3d package uses) it allows you, as a user, to edit the mesh with operations like bevel or select edge loop without the problems split vertices would give you.


    so from the article you posted, http://www.ericchadwick.com/examples/provost/byf2.html

    byf2_figure4.jpg

    FIGURE 4. Flat shading causes every face in a mesh to belong to a separate smoothing group, causing a worst-case split scenario to occur. Avoid at all costs unless specifically supported.


    I could make another diagram if that isn't clear, and also I haven't any actual experiences in coding a 3d engine so what I'm saying is not guaranteed to be 100% accurate :poly122:
  • Bertmac
    Options
    Offline / Send Message
    Bertmac polycounter lvl 17
    I also have problems with this,
    And have been reading up about it. but still:poly127:

    I did a normal map with a gun with lots of hard edges. all the hard edges where also split in the uv with a min of 12 pix of overlapping/ bleeding space. but still i had normal map hard edge problems + one more uv eiland

    here is a pic so you can see it .
    Hardedgetest.jpg

    Once somebody told me to smooth or average everything, but then i get black splotches

    getting lost here

    (model is from maya, normal from Xnormal, renders in Toolbag)
  • cptSwing
    Options
    Offline / Send Message
    cptSwing polycounter lvl 11
    ah yes, we're in full agreement here.. in my mind i'd moved away from the flatshaded model in the first post already ..
  • Frankie
    Options
    Offline / Send Message
    Frankie polycounter lvl 19
    cptswing: ahh, sorry I should have said more in my first post.

    Bertmac: what should it look like? It would probably be a lot easier to help if you posted the full screenshots of high/low/normal map, maybe in your thread? I don't know for sure but it looks like you're editing the model (even adjusting the smoothing counts) after you've baked the normal maps?
  • Bertmac
    Options
    Offline / Send Message
    Bertmac polycounter lvl 17
    Sorry didn't want to hijack the thread.
    I just had the same problem as the OP and tried some of the solutions handed in this thread, but so far no luck.
  • HAWK12HT
    Options
    Offline / Send Message
    HAWK12HT polycounter lvl 12
    Hi, I think you missed this tutorial :D

    Its for Max but some good tips in there.

    http://cg.tutsplus.com/tutorials/autodesk-3d-studio-max/how-to-bake-a-flawless-normal-map-in-3ds-max/

    As for that model you have i dont know what it is but if i were to make it i would clean up the area around holes as in low poly with Smooth groups those uneven surface will give nasty shading errors.
  • HAWK12HT
    Options
    Offline / Send Message
    HAWK12HT polycounter lvl 12
    Also add more polys (not too many :D ) on curved angles so SG can work better hence more good results from NM.

    SG= Smooth Group
    NM= Normal Map

    :D
  • EarthQuake
    Options
    Offline / Send Message
    Frankie wrote: »
    Not really, a uv split would split 1 vert into 2 where as most of them visible on the model could be split between 4 or 8 verts depending on edge direction, there's even a few cases where 10 verts would be possible. Well I think so anyway, I could be wrong...

    If your hard edge is along a uv border, it doesn't get split "again" the "real" vert count of your model in a game engine will always match the vertex count of your uvs, if not, your exporter is doing something funky. There is a max script somewhere that will count your actual vert usage.

    I mean, of course it is a terrible idea to have every edge on your model set to hard, but if every face on your model is uv mapped uniquely too, there isn't an extra cost to have those hard edges.

    In a realistic scenario, with a normal mesh and unwrap, there are only benefits to using hard edges along uv borders, and no real drawbacks.
  • EarthQuake
    Options
    Offline / Send Message
    Bertmac wrote: »
    I also have problems with this,
    And have been reading up about it. but still:poly127:

    I did a normal map with a gun with lots of hard edges. all the hard edges where also split in the uv with a min of 12 pix of overlapping/ bleeding space. but still i had normal map hard edge problems + one more uv eiland

    here is a pic so you can see it .
    Hardedgetest.jpg

    Once somebody told me to smooth or average everything, but then i get black splotches

    getting lost here

    (model is from maya, normal from Xnormal, renders in Toolbag)

    Your problem here is that your cage(envelope in maya) isn't averaged. This is a simple setting in the transfer maps options, its like... "Match using" geometry normals or mesh normals or something, just turn it to the other option. I can never remember which it is but one will give you an averged(seamless) projection while the other will have hard breaks in the projection.

    You may need to use a few extra verts in places when using an averaged projection to counter the skewing/waviness type artifacts that you will get though, but thats all just part of a good workflow. More here:

    http://www.polycount.com/forum/showthread.php?t=81154

    You got "black splotches"(smoothing errors most likely) because you averaged your mesh normals, not your projection mesh. Smoothing errors are caused by minor discrepancies between how the baker(maya) bakes and how the renderer(marmoset) displays the result. For best results you want to use a sync'd workflow, like displaying Maya bakes in Maya's viewport, or Max bakes with 3ps shader with QM.
  • Frankie
    Options
    Offline / Send Message
    Frankie polycounter lvl 19
    EarthQuake wrote: »
    If your hard edge is along a uv border, it doesn't get split "again" the "real" vert count of your model in a game engine will always match the vertex count of your uvs, if not, your exporter is doing something funky. There is a max script somewhere that will count your actual vert usage.

    Eq this is not correct. Well, I don't think so anway.

    vertsplits.jpg

    It would be possible to get the same vert count as the flat shaded example by making each triangle it's own uv island but I don't think I've ever seen someone do that accidently, where as I have come across lots of examples of people unnecessarily flat shading models, a building made with tiling textures would be a good example where someone wanted hard on all the corners and just set it to flat shaded.
  • m4dcow
    Options
    Offline / Send Message
    m4dcow interpolator
    Frankie wrote: »
    Eq this is not correct. Well, I don't think so anway.

    vertsplits.jpg

    It would be possible to get the same vert count as the flat shaded example by making each triangle it's own uv island but I don't think I've ever seen someone do that accidently, where as I have come across lots of examples of people unnecessarily flat shading models, a building made with tiling textures would be a good example where someone wanted hard on all the corners and just set it to flat shaded.

    I think EQ just mean't that the vertex counts won't increase if you make a hard edge in a spot where there is a UV seam already and vice versa.

    Calculating the vertex count takes in multiple factors, but he is right in saying that the closest number to it is usually the UV count. Remember each UV channel also adds to the vert count, so a mesh that is faceted in both UV and vertex normal (or flat shaded as you say) with 3 UV channels, will have that center point count for 32 vertices because of the UV splits.
  • Frankie
    Options
    Offline / Send Message
    Frankie polycounter lvl 19
    How did you get 32? I would have said 3*8=24.

    TBH I've avoided mentioning anything about multiple UV sets as I don't know enough about them. If I was guessing I'd say extra UVs are handled differently as they probably don't influence the bitangent calculations.

    UV counts ARE closer to the real number than XYZ counts but only because they are almost certainly going to be higher.
  • m4dcow
    Options
    Offline / Send Message
    m4dcow interpolator
    Frankie wrote: »
    How did you get 32? I would have said 3*8=24.

    Sorry I mean't 24. :P

    I don't know exactly how the extra UV sets are handled either, but the magazine article you linked to mentions extra UV sets briefly.

    "Moreover, if you have objects with multiple UV channels, the splits will occur successively through every channel."
  • Bertmac
    Options
    Offline / Send Message
    Bertmac polycounter lvl 17
    EarthQuake wrote: »
    Your problem here is that your cage(envelope in maya) isn't averaged. This is a simple setting in the transfer maps options, its like... "Match using" geometry normals or mesh normals or something, just turn it to the other option. I can never remember which it is but one will give you an averged(seamless) projection while the other will have hard breaks in the projection.

    You may need to use a few extra verts in places when using an averaged projection to counter the skewing/waviness type artifacts that you will get though, but thats all just part of a good workflow. More here:

    http://www.polycount.com/forum/showthread.php?t=81154

    You got "black splotches"(smoothing errors most likely) because you averaged your mesh normals, not your projection mesh. Smoothing errors are caused by minor discrepancies between how the baker(maya) bakes and how the renderer(marmoset) displays the result. For best results you want to use a sync'd workflow, like displaying Maya bakes in Maya's viewport, or Max bakes with 3ps shader with QM.

    Thanks Eq that was an nice piece of text to read. although i mostly don't bake my maps with maya i guess it would apply to Xnormal to.
    Should i export an cage from maya (the red one) and use it in Xnormal

    The link is very nice to, that USED to be a problem for me to thanks to clear that up . nice read
    Thanks
  • EarthQuake
    Options
    Offline / Send Message
    Bertmac wrote: »
    Thanks Eq that was an nice piece of text to read. although i mostly don't bake my maps with maya i guess it would apply to Xnormal to.
    Should i export an cage from maya (the red one) and use it in Xnormal

    The link is very nice to, that USED to be a problem for me to thanks to clear that up . nice read
    Thanks


    Yeah in XN you'll want to export a cage, or created a cage within the 3d viewer. If you're using the default ray distance settings in XN you'll get the broken edge problems too.
  • EarthQuake
    Options
    Offline / Send Message
    m4dcow wrote: »
    I think EQ just mean't that the vertex counts won't increase if you make a hard edge in a spot where there is a UV seam already and vice versa.

    Right, I said this in every post so far relating to the subject, c'mon Frankie, reading comprehension bro.
  • Frankie
    Options
    Offline / Send Message
    Frankie polycounter lvl 19
    My first point in the thread was

    "As well as the visual artifacts unnecessary split normals can cause it's also worth knowing that as it's essentially duplicating the vertex so it's hugely inefficient when you put it in game."

    relating to this picture

    Untitled-2.jpg

    where everything is clearly flat shaded or in other words every single vertex is split.


    I think its good advice for a beginner to not flat shade models for efficiency reasons.

    Not sure where I missed the point...
  • EarthQuake
    Options
    Offline / Send Message
    Frankie wrote: »
    My first point in the thread was

    "As well as the visual artifacts unnecessary split normals can cause it's also worth knowing that as it's essentially duplicating the vertex so it's hugely inefficient when you put it in game."

    relating to this picture

    Untitled-2.jpg

    where everything is clearly flat shaded or in other words every single vertex is split.


    I think its good advice for a beginner to not flat shade models for efficiency reasons.

    Not sure where I missed the point...
    EarthQuake wrote: »
    I mean, of course it is a terrible idea to have every edge on your model set to hard


    Probably when I agreed with you. =P

    There are two valid points to be made here:

    A. Using a faceted mesh/hard edges on every face is a bad idea. Both you and I have made this point.

    B. Using hard edges along your uv boarders on a normal, well optimized mesh has no drawbacks. - You keep bringing up A whenever someone brings up B, but they are two distinct points, which are both equally valid. Point B is not contrary to point A.

    Point A is of course the most beneficial to the OP. However, point B is often misunderstood, as many artists think "my whole model should be on 1SG" because it is "more optimized" or something, which simply isn't the case. So it is very relevant to a discussion about real vertex use.

    The advice I would give the op is: Set everything to one SG, then use a script to assign hard edges/sgs to your uv islands/borders. Renderhjs' Textools script has an easy to use function to do this. So again, not contrary advice to yours, simply expanded.
  • Bertmac
    Options
    Offline / Send Message
    Bertmac polycounter lvl 17
    Thanks EQ
    I am testing right now ( mainly because of the other thread you gave the link from )
    Its so much more clear now.
    thank you
  • Frankie
    Options
    Offline / Send Message
    Frankie polycounter lvl 19
    Yeah OK I mostly agree with that post :D
  • EarthQuake
    Options
    Offline / Send Message
    Frankie wrote: »
    Yeah OK I mostly agree with that post :D


    =D

    Did you ever finish that strap-on nazi chick?
  • Frankie
    Options
    Offline / Send Message
    Frankie polycounter lvl 19
    HAHA I don't think so. Maybe I've got the model hanging around somewhere if you want to take over?
  • EarthQuake
    Options
    Offline / Send Message
    No no, you should post it in pimping just for fun though =P
  • Frankie
    Options
    Offline / Send Message
    Frankie polycounter lvl 19
    polycount needs a "what were you working on then"

    shit I just found all my old stuff, got back to 1998 so far when I was 14!
  • EarthQuake
    Options
    Offline / Send Message
    Frankie wrote: »
    polycount needs a "what were you working on then"

    shit I just found all my old stuff, got back to 1998 so far when I was 14!

    That actually sounds like an awesome idea, feel free to start that thread.
  • R6Rider09
    Options
    Offline / Send Message
    Thanks again to all that posted. No worries about hijacking or anything, it's all good info. I appreciate the small debates on this kinda thing, as well. It give me a lot of insight on how to 'see' what I'm supposed to be doing.
  • Alberto Rdrgz
    Options
    Offline / Send Message
    Alberto Rdrgz polycounter lvl 9
    make that thing one smoothing group, and beef up those edges in your HP and that should do it.
  • K47
    Options
    Offline / Send Message
    K47
    Simple question:

    How do i average my cage in xNormal?

    Because everybody`s talking about averaging cages but this lil magic trick doesn`t happen to me.

    Here`s an image

    11bt6qv.png
Sign In or Register to comment.