Home Technical Talk

Understanding averaged normals and ray projection/Who put waviness in my normal map?

1235711

Replies

  • Sean VanGorder
    ikonane wrote: »
    Does anyone know what to do when "shaded" in projection mapping - cage does not work - how to fix?

    I've found that the shaded option won't work correctly if you are using hardware rendering (shadows/AO) in the viewport. Make sure that it's turned off.
  • Pac_187
    Offline / Send Message
    Pac_187 polycounter lvl 11
    You could also easily fix this with Photoshop.
    Only thing you have to do for this is to change the UV-Layout.

    Pics:
    Before
    before_fix.jpg

    After
    after_fix.jpg

    Old Normal Map
    oldnormal.jpg

    Using the "Smudge" Tool in Photoshop with a strength of about 50%
    fixednormal.jpg
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    Pac_187 wrote: »
    You could also easily fix this with Photoshop.
    Only thing you have to do for this is to change the UV-Layout.
    After
    Old Normal Map
    Using the "Smudge" Tool in Photoshop with a strength of about 50%

    The whole point of this thread is to teach people why the waviness happens and how to fix it without resulting to hacks like the smudge tool ;)

    Sure, things like that work to a certain extent, but its laborious because not only do you have to fix the normal...you need to do it on AO too. Then we have multiple bakes and revisions. What if someone else needs to work on something you did and then has too rebake?

    Its much more efficient and attractive to increase the number of sides :)
  • Pac_187
    Offline / Send Message
    Pac_187 polycounter lvl 11
    Okay, thanks for clarification :)
  • OddEyes
    Awesome thread.
    just thought to push it, cuz it contains a shitload of information
  • Isaiah Sherman
    Offline / Send Message
    Isaiah Sherman polycounter lvl 14
    I'm glad this is stickied. I was having this problem a few years back and it was blowing my mind. This is very important info, especially for people learning their first few bakes.

    Thanks!
  • pinkbox
    EDIT

    Nevermind ive gone back through this thread and learned a few things :poly142:
  • Hang10
    Wicked helpful, thank you to all! :D
  • AlexCatMasterSupreme
    Offline / Send Message
    AlexCatMasterSupreme interpolator
    Okay, I read through this entire thread, I am a noob, and a lot of it kinda confused me as well as I didn't understand a few things.
    By hard edge, this means two smoothing groups, correct? On say a 90 degree angle.

    2. I am really struggling with this bake: 5Htx0.jpg

    I have been at it for hours and tried everything I can think, it just will not bake right, I can make so much more complex objects but I can't seem to do this:/ I tried even making a higher poly one, I did a test bake, one with 28 sides, and one with 18, both had similar amounts of wavyness. That isn't as big an issue as my top part not working, I have literally been at this for hours, I must be tired or retarded or something. I have tried one smoothing group also. I am truly at a loss.
    Those edges at the top I have tried making the high so the lip is thicker and ugh I just can't think. I need sleep.
    It's just about the worse thing I have ever made ever. Here are the meshes lwPkz.png
    oIieM.png
    RswDt.png
    uqd1B.png

    I have tried about 6 different varations or more.
    I really tried to understand the welth of information presented to me in this thread, but I didn't even fully comprehend the first post to be honest.

    I am just getting into baking normal maps. I had my first bake where I actually put effort into learning and to learning to texture and it turned out alright, but this... Since I don't really have a poly limit for this prop (at all) I'm just making a super retarded 30 sided one, Unless someone can tell me how I can fix my other one. I know I seem like I'm dumping my issues, but I truely tried to understand, I really did. I am super new to baking normal maps.
    If someone has any advice I would be grateful.
  • Eric Chadwick
    I really tried to understand the welth of information presented to me in this thread, but I didn't even fully comprehend the first post to be honest.

    This is why you're having trouble.

    In the 1st image, EQ is explaining that a highpoly model works better for normalmapping when it has more slope for its bevels. The two cylinders at the left/front have very vertical bevels, so they bake poorly. Your barrel highpoly suffers because of this.

    This post goes into more detail about that point. You're always going to get some waviness with cylinders, but you can reduce that.

    Another issue with your barrel, the lowpoly model has too many indents in the middle area. These are unnecessary. The wireframes in this post, the ones marked 212 and 244, show how to solve this... don't model every highpoly dip into your lowpoly model. Most of the insets on your highpoly barrel can be flat in the lowpoly.

    Feel free to ask more questions. But also, spend some more time examining his images, and trying to understand what's different between the examples. There's a wealth of information there.
  • AlexCatMasterSupreme
    Offline / Send Message
    AlexCatMasterSupreme interpolator
    This is why you're having trouble.

    In the 1st image, EQ is explaining that a highpoly model works better for normalmapping when it has more slope for its bevels. The two cylinders at the left/front have very vertical bevels, so they bake poorly. Your barrel highpoly suffers because of this.

    This post goes into more detail about that point. You're always going to get some waviness with cylinders, but you can reduce that.

    Another issue with your barrel, the lowpoly model has too many indents in the middle area. These are unnecessary. The wireframes in this post, the ones marked 212 and 244, show how to solve this... don't model every highpoly dip into your lowpoly model. Most of the insets on your highpoly barrel can be flat in the lowpoly.

    Feel free to ask more questions. But also, spend some more time examining his images, and trying to understand what's different between the examples. There's a wealth of information there.

    I normally wouldn't leave all the indents in, but I would like to have them for some of the parts due to a band that wraps around a set of barrels I had modeled originally, I want that inset slightly.

    I have read that post several times, but I don't want to bevel the top and bottom of the barrel to remove this: 2115vr5.png

    I don't have any other 90 degree angles, like in that image of A, so I'm not trying to bake like that, I understand what he is saying, but besides leaving my indents modeled, I don't see how that is an issue. I mean, I would like them too, the prop doesn't have a poly limit so I don't really wanna make it lower. If I have to in order to make it bake better that's fair enough, but I still don't get what is causing my bake to have this artifact. I actually don't really have much waviness in my map at all it's that top region that is really giving me the trouble.
  • Eric Chadwick
    No limit? Then add more geometry, use a cylinder with 128 sides.
  • AlexCatMasterSupreme
    Offline / Send Message
    AlexCatMasterSupreme interpolator
    No limit? Then add more geometry, use a cylinder with 128 sides.
    I could, but at that point it would be pointless, I was thinking 30. Basically it's for a mod using x3 680's, by the time it's done it will be using the next generation of cards, so the limit is pretty damn high. It's not really to be used on anything lower. Like I said, the waviness wasn't my real issue, I thought it was at first, I actually didn't have really any noticeable waves.
  • Eric Chadwick
    The waviness will always be there, unless you match the resolution, which defeats the purpose. All you can do is minimize it. The best way to do this is to use more sides on your cylinder. If that takes you above the resolution you want to go, then you have to remove the triangles from other parts of the model.

    Frankly, this asset is badly designed for normal mapping. It has a lot of 90 degree angles, and a ton of insets. The little cap on top has too much detail. And the proportions seem odd. You don't need so much detail for a single prop. Unless you are a housefly in the game, and the gameplay is all about flying up close around this barrel.
  • AlexCatMasterSupreme
    Offline / Send Message
    AlexCatMasterSupreme interpolator
    The waviness will always be there, unless you match the resolution, which defeats the purpose. All you can do is minimize it. The best way to do this is to use more sides on your cylinder. If that takes you above the resolution you want to go, then you have to remove the triangles from other parts of the model.

    Frankly, this asset is badly designed for normal mapping. It has a lot of 90 degree angles, and a ton of insets. The little cap on top has too much detail. And the proportions seem odd. You don't need so much detail for a single prop. Unless you are a housefly in the game, and the gameplay is all about flying up close around this barrel.

    There is actually only two 90 degree angles, the top and bottom edges, I know it has a lot of detail for one prop. That's fine, that's the point, I'm not even done with it yet. Like I said, it's not giving my waviness, that's not really the issue at that point, its that separation at the top of the barrel that I have pointed out in red. I understand more sides reduces waviness, I am not having waviness issues... I thought I was, but I want to fix the top, that's all. I want to keep the insets, I'm not worried about the polycount of the current prop, and as I said I don't have a strict limit at all. I just want nice smooth edges on the insets.

    The top cap will be fine, I just want to know how to fix this top (pointed out in red) technically speaking, not talk about if my design is good enough, I have a ref, from Halo Reach, and this is a Halo mod, it has that detail, I'm adding that detail.

    Not trying to sound like I don't want to listen to any advice, it's just that I want to leave those in, and I explained why. As for the sizes of everything, I agree, I will fix that.
  • Eric Chadwick
    The seam is probably because your rendering method uses a different tangent basis than your baking method. What software are you using to bake the normal map from the high? And what software and method are you using for rendering the normal-mapped lowpoly asset?
  • AlexCatMasterSupreme
    Offline / Send Message
    AlexCatMasterSupreme interpolator
    I'm using 3ds max, and I have the low and high matched and I'm just baking it, I guess, like I always do, I have never really had this issue. I have the scene file here. http://dl.dropbox.com/u/82963315/Barrels.max
    It's messy, I pretty much dissected the scene 1000 times trying to figure this out.
  • Eric Chadwick
    What shader are you using on the lowpoly? Have you tried the 3Point Shader? http://www.polycount.com/forum/showthread.php?t=72861
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    @AlexCatMasterSupreme,

    You might want to try scaling in the main body of your barrel so that it more closely matches the HP, as currently it's baking the gap between the HP and the LP, so you need to reduce that gap. Of course having more sides would reduce the gap, but it depends how many sides you want to use. I scaled it in by 1.5%.

    barrel_scaled.jpg

    I baked the scaled barrel and got these results when viewed in Marmomset and in Blender.

    Marmoset
    barrel_marmoset.jpg

    Blender 2.63
    Barrel_Blender.jpg

    Map baked in xNormal 3.17.13
    Barrel_normals.jpg

    Also I have to agree with Eric that your mesh is really over the top for what it is. 1370 tris is pretty crazy for a barrel and you could get exactly the same visual result and if you are going to increase it to 30 side, then you are heading into the 2k range. Add the other 4 barrels and the metal strips and then that 2k becomes 8.5k or so, which is pretty nuts, especially when you could get the same visual result for much less than half of that (even with 30 sides).

    I understand that this is for next gen cards and that geo is cheap, but there is no point in using polys just because you can. If you keep the same frugal mindset when it comes to materials and geometry as we all do for the current gen (with a little added) by only using the geometry that you need, then it means that you can add more objects and textures to make the next gen scenes even more amazing.

    Anyway, it's your call...I just wanted to give my thoughts. :)
  • EarthQuake
    I'm using 3ds max, and I have the low and high matched and I'm just baking it, I guess, like I always do, I have never really had this issue. I have the scene file here. http://dl.dropbox.com/u/82963315/Barrels.max
    It's messy, I pretty much dissected the scene 1000 times trying to figure this out.

    Real quick Q: Baking with the cage, or "offset" method in max? If using offset, and if you have the top and bottom on separate smoothing groups, you'll get seams.
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    EarthQuake wrote: »
    Real quick Q: Baking with the cage, or "offset" method in max? If using offset, and if you have the top and bottom on separate smoothing groups, you'll get seams.
    He's baking with a cage by the looks of it, EQ.

    RswDt.png
  • AlexCatMasterSupreme
    Offline / Send Message
    AlexCatMasterSupreme interpolator
    I did try scaling it, and I thought that was the issue, but it never turned out right, but thanks a lot guys, seriously, and I know that it's a large polycount, and if it wasn't for this mod I wouldn't go over the top with the geo, but yeah, in this one case it's okay,Basically, the point of the project is to see how far we can push Cryengine with cards coming out in the next year or two, not really to make it playable on even a 2000 dollar machine. I'm going to work on the design some more, but now that I know for sure what the issue is I feel a lot better, it was really irritating me.

    So thanks very very much guys. I'm gunna go get on it right now:)
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    Ahh, cool :)

    Have fun with the mod!
  • probiner
    Offline / Send Message
    probiner polycounter lvl 12
    Would just like to thank once again to EQ and other contributors for this thread. From time to time I keep coming back and slap my forehead.

    Cheers
  • fearian
    Offline / Send Message
    fearian greentooth
    Hey guys I have a quick question somewhat related to this subject:

    What are peoples views on bumping up polycounts for portfolio peices. By adding a couple of loops I can get a good bake, but by adding a bunch I can get something indistinguishable from the high poly. For a 'hero prop' where the goal is something closer to a study, should I just throw polycounts out the window?
  • fearian
    Offline / Send Message
    fearian greentooth
    Obviously; but in a portfolio setting, where do you draw the line? I could add geo that does nothing for the silhouette of an object, but merely makes a surface shade flat rather than *slightly* rounded. is this just being anal? The bake was good enough for production before, and going into this kind of perfectionism across the board will take a large model from 7k to 10k.
  • Racer445
    Offline / Send Message
    Racer445 polycounter lvl 12
    in my experience an extremely clean bake will outweigh tricount for portfolios. most of the times once you add edges to get your bake looking like you want it, you can figure out where to remove geo to keep the tricount down. a flawless bake can often be only a few hundred more tris, or even the same amount as budgeted.

    designing your model for baking is a huge part too.
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    fearian wrote: »
    Obviously; but in a portfolio setting, where do you draw the line? I could add geo that does nothing for the silhouette of an object, but merely makes a surface shade flat rather than *slightly* rounded. is this just being anal? The bake was good enough for production before, and going into this kind of perfectionism across the board will take a large model from 7k to 10k.

    Just try and be proportionate with the extra geometry that you are adding. Increasing a models tri count by 50% to make it *slightly* better is probably a bit over the top and depending on the object you could most likely get the same effect by making a separate smoothing group from that part with only the cost of the extra verts. around the edge, which might be cheaper.

    Add more tris because you need them, not because you can add more tris. :)
  • fearian
    Offline / Send Message
    fearian greentooth
    Thanks guys. I think in my case it is totally over the top, given the cost to result ratio. I'll be posting new screens soon in my wip thread, but I'm making a lowpoly and bake for this vault door I did. Loooots of cylinders to play with :D
  • ZacD
    Offline / Send Message
    ZacD ngon master
    I want to make sure I'm understanding out to make a cage for xnormal for a cylinder-ish shaped. I just take the lowpoly and edit it to be a cylinder that has flat sides? Or can it be any mesh, I believe it'd need to have the same UV's as the low poly. I'd also assume it'd need to be pushed, correct?

    Basically I'm baking a bottle, which should be similar enough to that camera example on the wiki.
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    ZacD wrote: »
    I want to make sure I'm understanding out to make a cage for xnormal for a cylinder-ish shaped. I just take the lowpoly and edit it to be a cylinder that has flat sides? Or can it be any mesh, I believe it'd need to have the same UV's as the low poly. I'd also assume it'd need to be pushed, correct?

    Basically I'm baking a bottle, which should be similar enough to that camera example on the wiki.

    If you are using a custom cage (an exported mesh) in xNormal you will need to have exactly the same topology and vert index/count for the cage to work or it won't let you bake.
    You can use the cage feature in xNormal though (in the 3d viewer) and edit it how you like, though if you are trying to remove waviness, you would be better just adding more sides to your low poly. :)
  • ZacD
    Offline / Send Message
    ZacD ngon master
    Okay, cool, I'll just duplicate the lowpoly, shrinkwrap it to a cylinder, and call it done. :)
  • Matildags
    if you paint it straight.
  • fatihG_
    Offline / Send Message
    fatihG_ polycounter lvl 14
    Just a quick question regarding cages in Maya.

    I have read multiple times that in Maya you can not adjust the cage, apart from just moving it along its normals. Though I have edited a cage in Maya multiple times, how I would edit a normal mesh. Am I missing something here? Maybe I just remember things not accurately.

    Afaik a cage is simply a duplicate of the mesh, kinda like a blend shape, with just different vertex positions right?
    So you could just use the default Maya cage method and export that mesh for xNormal, instead of manually duplicating your mesh and what not. Though I must say that I have not tried baking with an edited cage in Maya.. as in Maya baking takes a damn long time and I just didnt have the patience/time to wait for it to finish. (Didn't set up a simple test scene, was trying to bake a project mesh)
  • timotronprime
    Offline / Send Message
    timotronprime polycounter lvl 11
    bb0x wrote: »
    Afaik a cage is simply a duplicate of the mesh, kinda like a blend shape, with just different vertex positions right?
    In addition to that, it should have all soft edges only (in Max terms, only 1 smoothing group).
    bb0x wrote: »
    So you could just use the default Maya cage method and export that mesh for xNormal, instead of manually duplicating your mesh and what not.
    I've not used the Maya Transfer Map since forever so I wouldn't know how the envelope for that work, but you cannot use your game-res mesh as a cage also for xNormal because it has explicit normals, and as far as I know, xNormal does not have a function to modify the cage's normals prior to baking.
  • filip
    Offline / Send Message
    filip polycounter lvl 9
    Hi!

    Please take a look at this image:
    7xzDwJe.jpg

    Help me understand how this stuff works... I think I'm too dumb to understand the tutorial in this topic. Do I understand correctly that I absolutely need to add extra edges on the low poly around the area where the inset screens are so they don't wave like that in the normal map? No other way?
    Also, all the edges are smooth on the low poly (in Max terms that would mean that all faces are part of the same smoothing group, I think) so the 90 degrees edges bake correctly. Is this the only way to correctly bake 90 degrees edges?

    And I don't understand the thing about separating UV islands.. doing that with hard 90 degrees edges still gave me seams on those edges. Only way I could fix it was by smoothing them.
  • Quack!
    Offline / Send Message
    Quack! polycounter lvl 17
    Do I understand correctly that I absolutely need to add extra edges on the low poly around the area where the inset screens are so they don't wave like that in the normal map? No other way?

    Yes. Your normals are getting pulled towards the verts, that is why they stretch. Another fix would be to add that detail using ndo2/crazybump in Photoshop.
    Also, all the edges are smooth on the low poly (in Max terms that would mean that all faces are part of the same smoothing group, I think) so the 90 degrees edges bake correctly. Is this the only way to correctly bake 90 degrees edges?

    90 degree edges will give problems no matter the workflow. However, with a synced workflow / handplane, you can get away with having 90 degree angles on your uv islands. However, you will always have seams at uv islands, they will be small but still present, but will be hidden by your diffuse / lighting. I recommend to just add geometry via a small bevel at 80-90 degree angles if you can affort the verts. This way you increase the visual fidelity of the silhouette of the object as well as ease the edge to make the bake work a bit better and it's almost always as cost effective as adding a break at that same edge.
    And I don't understand the thing about separating UV islands.. doing that with hard 90 degrees edges still gave me seams on those edges. Only way I could fix it was by smoothing them.

    Set your smoothing groups / hard edges to your uv islands, usually with a script so you don't do it by hand,. Plain and simple. After you have done this you go in and fix smoothing groups based on the shape of your object.

    If you have 1 smoothing group and uv islands on an edge you will get highly visible seam.
  • m4dcow
  • WarrenM
    m4dcow has the right of it. Watch the first 2 videos here for the best and most concise explanation I've ever seen on this topic:

    http://www.handplane3d.com/videos.html

    "What are normal maps?"
    "Controlling shading"

    These made everything click for me and now my normal maps are beautiful. :)
  • Dantert
    Offline / Send Message
    Dantert polycounter lvl 10
    This was really helpful!! Thanks!!
  • MooreCreativity
    I'm having an issue with my normal maps. I'm trying to understand if it's an issue with the model/the uvws or what. I am using 3Ds Max to import an obj into from Mudbox. It doesn't seem to be displaying the model the same in Max as it did in Mudbox. Also, the seams are showing up in Max. Can someone help me figure out what I'm doing wrong? I really want to learn this stuff. Thanks! Please let me know if you need to see any other screen shots or anything. Normal.pngDaschund https://www.dropbox.com/lightbox/home/Daschund, https://www.dropbox.com/s/ysk1r96r73ngplc/Normal.png.
  • another caveman
    Offline / Send Message
    another caveman greentooth
    Hi!
    I'm having the same kind of problems, I've read the main post of this thread, and I still can't figure out how to fix this, I've tried to move / add edges to my lowpoly, but nothing seems to work


    HtW8N74.png

    And I keep rendering this kind of "artifacts" I don't really know how I should call it since I got no idea what it is:


    WuQxf1j.png

    So: Does it come from my UV mapping? Does it come from my render setups? Or does it come from my topology or w/e?

    Thanks
  • Nosslak
    Offline / Send Message
    Nosslak polycounter lvl 12
    You need to use a mesh with averaged/smoothed normals (you can use Handplane here) or a cage while you bake to fix this.

    The topology and UVs look pretty reasonable. You might want some more space between the islands as that would make it look better when the textures get mipped/downscaled, but otherwise it looks alright.
  • another caveman
    Offline / Send Message
    another caveman greentooth
    Thanks for your answer, however, I'm using a projection cage already (so wut now?) I've been trying to find similar problems on the web but nothing came out :/
  • EarthQuake
    Spaggiari wrote: »
    Thanks for your answer, however, I'm using a projection cage already (so wut now?) I've been trying to find similar problems on the web but nothing came out :/

    Need more info

    What app are you baking in?

    What app/engine is it going to?

    What does your lowpoly look like?

    What do your uvs look like?

    What does your high look like?

    etc
  • another caveman
    Offline / Send Message
    another caveman greentooth
    I'm using 3ds max, my model's staying in 3ds max / marmoset, it's just some exercise you know.

    high / low

    esUgIHU.png

    It's just a simple suppressor.

    UVs:

    kgJFeZe.png
  • BARDLER
    Offline / Send Message
    BARDLER polycounter lvl 12
    It looks like your edges are way to sharp. The normal map is baking 0 to 1 pixel for your edges which are making it render weird.
    NormalMap?action=AttachFile&do=get&target=normal_edge_thickness.jpg
  • another caveman
    Offline / Send Message
    another caveman greentooth
    Well I've just tried to remove those sharp edges, I still get the same result. That's bothering.
    PS: that's not ONLY with that suppressor model, it happens on every & each model I try to bake, something must be wrong with my setups?
  • EarthQuake
    Spaggiari wrote: »
    Well I've just tried to remove those sharp edges, I still get the same result. That's bothering.
    PS: that's not ONLY with that suppressor model, it happens on every & each model I try to bake, something must be wrong with my setups?

    If you bake with max you'll have to invert the green channel when loading into Marmoset Toolbag. The wrong "handness" or "swizzle" will cause lighting artifacts and seams galore.

    Also as noted, your edges on your highpoly are way too tight, you would need a crazy high resolution texture to represent those tiny bevels.
  • another caveman
    Offline / Send Message
    another caveman greentooth
    I'm not rendering in marmoset yet, those black things appears in max when I load my normals on my lowpo model.
    And yeah, I tried to remove those edges, it didn't work.

    edit:

    9br4.png

    I just unticked this case and now it does work!
1235711
Sign In or Register to comment.