Home Technical Talk

The Maya Cylinder Normal Map Problem

polycounter lvl 14
Offline / Send Message
Isaiah Sherman polycounter lvl 14
So I have been wanting to bake some normal maps for a western styled six-shooter but I'm getting some issues around cylinder edges.

Before you jump the gun and start throwing links of the Polycount Normal Map wiki, End All Be All Normal Map thread, Poopinmymouth's normal map tutorial, and the 1.05MB SWF file on Maya normal maps... I've seen it all. All day yesterday (Sunday) was dedicated to reading things trying to figure this out.

The only thing that came remotely close to actually solving this issue in Maya was the last thing I mentioned.

cylinderNormalMapIssue.jpg

Replies

  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    It's a problem on any geometry where you have a face bounded by two sets of 90 degree interior angles.

    As you found out it's the ray cast direction from the normals that results in either "wavy" edges, or ray misses if you have hard edges.

    I tend to go for the following solution - just a single bevel on the lowpoly at 45 degrees (like your example but only one division on the bevel instead of 2). Then you just map the entire cap piece (including the bevel) planar from the top, and unwrap the sides. Should avoid any waviness and still keep the nice rounded edges.
  • airbrush
    Offline / Send Message
    airbrush polycounter lvl 13
    This is from a similar thread I replied to a couple years back regarding these kinds of issues, baking of low poly cylindrical and spherical type objects...

    http://boards.polycount.net/showpost.php?p=761510&postcount=8
  • EarthQuake
    Bevel the edge, or in some cases if you increase the number of sides it will fix the problem. Increasing the amount of sides can be ideal, because a 24 sided cylinder is the same data as a 12 side cylinder with 2 beveled edges.

    The reason increasing the sides works, is that there is less difference between the surface of the high and the low, which is really what you're seeing when you get "wavyness" its the difference between the two meshes when raytraced from an angle.

    [edit]

    Heres an image to show what i mean:

    cylinder_sides.jpg

    You see here just adding sides is clearly the best method. Because it matches the HP the closest. Beveling the edges here A. Doesnt match the shape as well and B. Creates some small-thin triangles that are best to avoid. Many many problems with baking can be fixed by simply making sure your LP matches your HP better.

    However, if you had say, a bunch of floating detail on these cylinders, those details would come out all skewed without some beveled edges or extra loops for supporting edges, so just adding extra sides wouldn't be ideal here. In that case would probabbly do something like 16 instead of 12, and loops/bevels for support.
  • Isaiah Sherman
    Offline / Send Message
    Isaiah Sherman polycounter lvl 14
    Still getting some pretty harsh waviness. I went off of EQ's example of the 24-sided cylinder. I am going to try the 45 degree bevel now.

    I included my map baking options to see if I have anything wrong in there. You can enlarge the image to read the text in the map baking window better.


    sadface.jpg
  • Ark
    Offline / Send Message
    Ark polycounter lvl 11
    Just paint it out in PS.
  • Isaiah Sherman
    Offline / Send Message
    Isaiah Sherman polycounter lvl 14
    I could for this example, but it would be tedious to paint it out when you have a 6-shooter that has a lot of edges all with this issue. I'd rather learn to do it right.
  • Isaiah Sherman
    Offline / Send Message
    Isaiah Sherman polycounter lvl 14
    Okay, I'm starting to think I'm doing something wrong in my bake options because this result was even more awful that before. In fact, it's the worst! Clearly something is going wrong here.

    I don't understand why the caps are getting such harsh lines, either. I had the whole object softened before unwrapping. Planar projection in Y on caps (and bevel faces) and a cylindrical on the rest. Unwrapped after beveled + history deleted.

    reallybad.jpg
  • pior
    Online / Send Message
    pior grand marshal polycounter
    Dude, it's normalmaps you're dealing with! It's no magic... it will always be an approximation. Make it look good from the angle it is seen the most.
  • MoP
    Offline / Send Message
    MoP polycounter lvl 18
    yeah :)
    waviness like that probably won't be noticed a lot of the time unless it's really huge and seen from straight-on the side all the time
  • EarthQuake
    for the 24 sided example, make sure that top and bottom caps of your mesh are exactly where they are in the HP, thats really the only thing i can think of, if its a little off its going to be rendering the edge in the wrong place.
  • Isaiah Sherman
    Offline / Send Message
    Isaiah Sherman polycounter lvl 14
    I SUPPOSE I'll have to live with it. I just wish everything in 3D could be magic, that would be great :)

    I believe for my project I will be going with the 24 sided cylinder because of how the gun will be laid out. So, I guess my question was answered... I was wondering if it actually WAS possible to completely remove it. Now that I know, I will just do my best to reduce it.

    Thanks a lot for your guys' help, I really appreciate it :)
  • pior
    Online / Send Message
    pior grand marshal polycounter
    Keep in mind that with a 24-sided cylinder, you will most likely get 24 ugly sides.

    ...

    ...

    :D
  • Isaiah Sherman
    Offline / Send Message
    Isaiah Sherman polycounter lvl 14
    Hahahahaha... noway!? You mean I can't normal map a low-poly cylinder that will completely get rid of all my life worries and let me live for eternity in happiness?

    Damnit... I was getting my hopes up :poly131:
  • Warhamster
    Haiasi wrote: »
    I SUPPOSE I'll have to live with it. I just wish everything in 3D could be magic, that would be great :)


    Hahaha, well too bad you are not into animation. In my opinion animating in 3D is basically magic. And thanks for getting a question that i also had answered! :)
  • kodde
    Offline / Send Message
    kodde polycounter lvl 18
    Is there such a thing as a perfect normal map for a lowpoly cylinder?

    I mean either you get a good "faked" roundness at a certain angle which makes it appear as though it has more sides that it really does (silhouette being the exception), but then you get the "wavy" lines when looking at other angles.

    If you manage to get rid of those "wavy" lines you wouldn't have the roundness that makes it seem to have have more sides than it really does?

    Am I wrong? Can't see how you could get both with a normalmap. You can fake a bevel without wavy lines, but faking more sides comes with wavy lines no?
  • kodde
    Offline / Send Message
    kodde polycounter lvl 18
    My thoughts in an hopefully explanatory image.

    cylindernmapsmoothing.jpg
  • Isaiah Sherman
    Offline / Send Message
    Isaiah Sherman polycounter lvl 14
    So what did you do to achieve the far right example? I see that the edges are jagged, but from the side you don't have wavy lines, which is what I'm after.

    All my examples / tests have the result of your middle cylinder.
  • kodde
    Offline / Send Message
    kodde polycounter lvl 18
    Well I suppose you could achieve this with a tweaked cage for projection, but I actually just extracted the caps so that I had 3 separate objects. Then I had to tweak the UVs slightly. I wouldn't call this a pretty way of doing it, but that's how I created the example one on the right.

    I didn't give it much thought, but using this way you are stuck with using hard edges I think.
  • Isaiah Sherman
    Offline / Send Message
    Isaiah Sherman polycounter lvl 14
    Tweaking the cage in Maya doesn't do anything for you since it doesn't adjust ray cast direction. All it does is use it as a reference for how far to cast the rays (typically just set at 100% inside only).

    I will try your 3 object bake test tomorrow. It's 4:15AM and I need to cuddle my girlfriend :)
  • bugo
    Offline / Send Message
    bugo polycounter lvl 17
    instead of doing a highpoly mesh smooth, consider getting your low poly and doing a bevel on the corners and using it as a high (dont mesh smooth). This will create what you want.
  • kodde
    Offline / Send Message
    kodde polycounter lvl 18
    Yeah, that should provide same result as my example on the right.
  • pior
    Online / Send Message
    pior grand marshal polycounter
    Hey H,

    unfortunately it seems like tweaking the cage in maya doesn't do much (it can even cause big problems when you simply display it to check its thickness - it's all broken), however, the different results that Kodde posted can be achieved in 3DSMax by positionning the cage verts where you want them (along the length of the cylinder, or along the surface of the cap). You might also be able to get the same variety of results using Xnormal, and importing the desired cage as obj (or tweaking the cage inside xnormal but iirc it's a bit of a convoluted problem)

    I usually shift-smudge in photoshop to get rid of unwanted waves. (I like the right hand side better most of the time)
  • EarthQuake
    That right side one with the perfectly straight lines looks worse to me about 90% of the time. =D You ruin the highpoly look for all but a very specific viewing angle, i really dont see the point and think its wasted effort to think about it this much.
  • Isaiah Sherman
    Offline / Send Message
    Isaiah Sherman polycounter lvl 14
    I was thinking that if I really wanted to take care of the issue I would have to use another program. I have Xnormal installed so exporting a cage sounds like a good idea. I'll be learning a lot about 3DS Max next term so I'm excited about that because I know Max has a lot of amazing tools.

    Thanks for the help, all of you. If I have any more questions I'll be sure to post it up :)
  • pior
    Online / Send Message
    pior grand marshal polycounter
    Hehe EQ depends if 90/100 of the cylinders end up in that angle or not :P
  • yurid9
    Offline / Send Message
    yurid9 polycounter lvl 10
    I was having the same problems. The solution I found was that you have to put edge loops near the ends of the cylinder. This usually takes care of the waviness problem. I hope it helps.
  • radiancef0rge
  • yurid9
    Offline / Send Message
    yurid9 polycounter lvl 10
    2009 brah

    Oh. Totally did not see the date there haha
  • latitudeclear
    Ahahaha. I found it to be an interesting thread anyways. Same technical limitations still plague us.
  • Isaiah Sherman
    Offline / Send Message
    Isaiah Sherman polycounter lvl 14
    I know this thread is old, but another way to avoid this is to harden the edges of the UV shell borders.

    I was still a student back in 2009 so I didn't know what I was doing :D


    ....hell... I still don't...
  • kodde
    Offline / Send Message
    kodde polycounter lvl 18
    There's no way around this issue still, hard edges of the UV or what not. If it's critical to look good in all angles, use more geometry spans. Otherwise choose the rounded look at the right angles, or stick with beveled edges at the top but have the "lowpoly" look of few spans. In my opinion, just choose the right balance/method for your specific case.
  • McClish
    Offline / Send Message
    McClish polycounter lvl 6
    Build and bake your cylinder flat then use a bend to get de cylinder shape back after + hard edge when you have seams.
  • kodde
    Offline / Send Message
    kodde polycounter lvl 18
    Here we go again :O

    I'd advice anyone doubting to just try all the options yourself. There's no perfect solution to magically make a low poly cylinder look high poly in all conditions(angles). 

    McClish> What you are suggesting sounds like the case of "beveled" top edge look if I understand you correctly? Would still not achieve the look of having more spans when viewed slightly from above/below.
Sign In or Register to comment.