The Maya Cylinder Normal Map Problem

Isaiah Sherman

So I have been wanting to bake some normal maps for a western styled six-shooter but I'm getting some issues around cylinder edges.

Before you jump the gun and start throwing links of the Polycount Normal Map wiki, End All Be All Normal Map thread, Poopinmymouth's normal map tutorial, and the 1.05MB SWF file on Maya normal maps... I've seen it all. All day yesterday (Sunday) was dedicated to reading things trying to figure this out.

The only thing that came remotely close to actually solving this issue in Maya was the last thing I mentioned.

MoP

It's a problem on any geometry where you have a face bounded by two sets of 90 degree interior angles.

As you found out it's the ray cast direction from the normals that results in either "wavy" edges, or ray misses if you have hard edges.

I tend to go for the following solution - just a single bevel on the lowpoly at 45 degrees (like your example but only one division on the bevel instead of 2). Then you just map the entire cap piece (including the bevel) planar from the top, and unwrap the sides. Should avoid any waviness and still keep the nice rounded edges.

airbrush

This is from a similar thread I replied to a couple years back regarding these kinds of issues, baking of low poly cylindrical and spherical type objects...

http://boards.polycount.net/showpost.php?p=761510&postcount=8

EarthQuake

Bevel the edge, or in some cases if you increase the number of sides it will fix the problem. Increasing the amount of sides can be ideal, because a 24 sided cylinder is the same data as a 12 side cylinder with 2 beveled edges.

The reason increasing the sides works, is that there is less difference between the surface of the high and the low, which is really what you're seeing when you get "wavyness" its the difference between the two meshes when raytraced from an angle.

[edit]

Heres an image to show what i mean:



You see here just adding sides is clearly the best method. Because it matches the HP the closest. Beveling the edges here A. Doesnt match the shape as well and B. Creates some small-thin triangles that are best to avoid. Many many problems with baking can be fixed by simply making sure your LP matches your HP better.

However, if you had say, a bunch of floating detail on these cylinders, those details would come out all skewed without some beveled edges or extra loops for supporting edges, so just adding extra sides wouldn't be ideal here. In that case would probabbly do something like 16 instead of 12, and loops/bevels for support.

Isaiah Sherman

Still getting some pretty harsh waviness. I went off of EQ's example of the 24-sided cylinder. I am going to try the 45 degree bevel now.

I included my map baking options to see if I have anything wrong in there. You can enlarge the image to read the text in the map baking window better.

Ark

Just paint it out in PS.

Isaiah Sherman

I could for this example, but it would be tedious to paint it out when you have a 6-shooter that has a lot of edges all with this issue. I'd rather learn to do it right.

Isaiah Sherman

Okay, I'm starting to think I'm doing something wrong in my bake options because this result was even more awful that before. In fact, it's the worst! Clearly something is going wrong here.

I don't understand why the caps are getting such harsh lines, either. I had the whole object softened before unwrapping. Planar projection in Y on caps (and bevel faces) and a cylindrical on the rest. Unwrapped after beveled + history deleted.

pior

Dude, it's normalmaps you're dealing with! It's no magic... it will always be an approximation. Make it look good from the angle it is seen the most.

MoP

yeah
waviness like that probably won't be noticed a lot of the time unless it's really huge and seen from straight-on the side all the time

EarthQuake

for the 24 sided example, make sure that top and bottom caps of your mesh are exactly where they are in the HP, thats really the only thing i can think of, if its a little off its going to be rendering the edge in the wrong place.

Isaiah Sherman

I SUPPOSE I'll have to live with it. I just wish everything in 3D could be magic, that would be great

I believe for my project I will be going with the 24 sided cylinder because of how the gun will be laid out. So, I guess my question was answered... I was wondering if it actually WAS possible to completely remove it. Now that I know, I will just do my best to reduce it.

Thanks a lot for your guys' help, I really appreciate it

pior

Keep in mind that with a 24-sided cylinder, you will most likely get 24 ugly sides.

...

...

Isaiah Sherman

Hahahahaha... noway!? You mean I can't normal map a low-poly cylinder that will completely get rid of all my life worries and let me live for eternity in happiness?

Damnit... I was getting my hopes up :poly131:

Warhamster

Haiasi wrote: »

I SUPPOSE I'll have to live with it. I just wish everything in 3D could be magic, that would be great

Hahaha, well too bad you are not into animation. In my opinion animating in 3D is basically magic. And thanks for getting a question that i also had answered!

kodde

Is there such a thing as a perfect normal map for a lowpoly cylinder?

I mean either you get a good "faked" roundness at a certain angle which makes it appear as though it has more sides that it really does (silhouette being the exception), but then you get the "wavy" lines when looking at other angles.

If you manage to get rid of those "wavy" lines you wouldn't have the roundness that makes it seem to have have more sides than it really does?

Am I wrong? Can't see how you could get both with a normalmap. You can fake a bevel without wavy lines, but faking more sides comes with wavy lines no?

kodde

My thoughts in an hopefully explanatory image.

Isaiah Sherman

So what did you do to achieve the far right example? I see that the edges are jagged, but from the side you don't have wavy lines, which is what I'm after.

All my examples / tests have the result of your middle cylinder.

kodde

Well I suppose you could achieve this with a tweaked cage for projection, but I actually just extracted the caps so that I had 3 separate objects. Then I had to tweak the UVs slightly. I wouldn't call this a pretty way of doing it, but that's how I created the example one on the right.

I didn't give it much thought, but using this way you are stuck with using hard edges I think.

Isaiah Sherman

Tweaking the cage in Maya doesn't do anything for you since it doesn't adjust ray cast direction. All it does is use it as a reference for how far to cast the rays (typically just set at 100% inside only).

I will try your 3 object bake test tomorrow. It's 4:15AM and I need to cuddle my girlfriend

bugo

instead of doing a highpoly mesh smooth, consider getting your low poly and doing a bevel on the corners and using it as a high (dont mesh smooth). This will create what you want.

kodde

Yeah, that should provide same result as my example on the right.

pior

Hey H,

unfortunately it seems like tweaking the cage in maya doesn't do much (it can even cause big problems when you simply display it to check its thickness - it's all broken), however, the different results that Kodde posted can be achieved in 3DSMax by positionning the cage verts where you want them (along the length of the cylinder, or along the surface of the cap). You might also be able to get the same variety of results using Xnormal, and importing the desired cage as obj (or tweaking the cage inside xnormal but iirc it's a bit of a convoluted problem)

I usually shift-smudge in photoshop to get rid of unwanted waves. (I like the right hand side better most of the time)

EarthQuake

That right side one with the perfectly straight lines looks worse to me about 90% of the time. =D You ruin the highpoly look for all but a very specific viewing angle, i really dont see the point and think its wasted effort to think about it this much.

Isaiah Sherman

I was thinking that if I really wanted to take care of the issue I would have to use another program. I have Xnormal installed so exporting a cage sounds like a good idea. I'll be learning a lot about 3DS Max next term so I'm excited about that because I know Max has a lot of amazing tools.

Thanks for the help, all of you. If I have any more questions I'll be sure to post it up

pior

Hehe EQ depends if 90/100 of the cylinders end up in that angle or not :P

yurid9

I was having the same problems. The solution I found was that you have to put edge loops near the ends of the cylinder. This usually takes care of the waviness problem. I hope it helps.

radiancef0rge

2009 brah

yurid9

radiancef0rge wrote: »

2009 brah

Oh. Totally did not see the date there haha

latitudeclear

Ahahaha. I found it to be an interesting thread anyways. Same technical limitations still plague us.

Isaiah Sherman

I know this thread is old, but another way to avoid this is to harden the edges of the UV shell borders.

I was still a student back in 2009 so I didn't know what I was doing


....hell... I still don't...

kodde

There's no way around this issue still, hard edges of the UV or what not. If it's critical to look good in all angles, use more geometry spans. Otherwise choose the rounded look at the right angles, or stick with beveled edges at the top but have the "lowpoly" look of few spans. In my opinion, just choose the right balance/method for your specific case.

McClish

Build and bake your cylinder flat then use a bend to get de cylinder shape back after + hard edge when you have seams.

kodde

Here we go again :O

I'd advice anyone doubting to just try all the options yourself. There's no perfect solution to magically make a low poly cylinder look high poly in all conditions(angles). 

McClish> What you are suggesting sounds like the case of "beveled" top edge look if I understand you correctly? Would still not achieve the look of having more spans when viewed slightly from above/below.