I looked through the normal map thread and did not see any screenshots like this. I thought this would have been a pretty easy thing to normal map but I can't figure out what I'm doing wrong. Tried everything, beveling the low res, different uv mapping. How do you guys fix this?
That's to be expected anytime your low poly has less sides than your high.
If there is a cage you can adjust in Maya try this, it will help minimize it but probably won't eliminate it.
You can also clean it up by hand a few ways.
Use the Filter > Liquify to nudge the shapes around. If you need to clean up multiple layers (like an AO bake also) save the cage and load it onto the other layers.
You can also select a single row of pixels on the edge of the cylinder unwrap and drag it out across the waves.
That's to be expected anytime your low poly has less sides than your high.
If there is a cage you can adjust in Maya try this, it will help minimize it but probably won't eliminate it.
You can also clean it up by hand a few ways.
Use the Filter > Liquify to nudge the shapes around. If you need to clean up multiple layers (like an AO bake also) save the cage and load it onto the other layers.
You can also select a single row of pixels on the edge of the cylinder unwrap and drag it out across the waves.
The first thing here is a max specific fix, maya's cage only adjusts ray distance, not ray direction, so tweaking the cage doesnt actually change anything.
The second solution is ok, but can be a real big pain in the ass if you need to edit multiple maps (n, ao, color bake, etc). Or if you have a complex uv layout, or your uv edges arent all straight.
So its best to instead try to understand WHY this happens, and fix the problem with your geometry at the start.
So, the reason WHY you get the wavy lines is this:
When you use a lowpoly mesh to project onto a high, it uses the vertex normals for the ray direction. Since your vertex normals on those areas where you have the beveled out midsection are pointing somewhere near 45*, you're not just projecting straight out. You get the wavy lines because you're seeing the difference between the high and the low. To solve this with geometry you have pretty much two options:
1. Make the vertex normals more accurate, you can do this by cutting in more support loops. You'll be adding extra loops that arent doing much or anything to the silhouette so this may not be the best solution, but to account for wavy lines/skewed details sometimes its a must.
2. Lessen the difference between the high and the low, the more closely your low matches to your high, the less "wavy difference" you'll get in your NM. So you could try 12 sides, or 16 sides to get a better result. This method generally results in an overall better asset.
Now of course you can't always just add more geometry, but that is often the best solution to many normal projection problems, be it fixing wavy lines, skewed details, or getting rid of pesky smoothing errors.
Some people get really anal about these wavy lines, but me unless they are very bad(your example is pretty bad) i wont worry about them unless they are on a very important/highly visible part of the asset, then again, adding the right amount of geometry there not only fixes the bake errors but gives a better overall asset.
In a lot of cases, depending on how you view the object, the wavy lines can actually help to make it appear more round, from certain angles, of course it can look much worse from certain angles as well, so it helps to be mindful of how the asset will be viewed. In fact the area you have circled actually appears to be more round than it actually is, which is a plus, however the bottom is obviously wavy. Really in few cases do i ever want to eliminate it entirely, because then you are just showcasing how lowres your mesh actually is, and the goal should be to emulate the look of the high more than anything, instead of going crazy over slight wavyness.
Thanks guys, yeah this shit looks fine from the 3/4 view but the object will mostly be seen from the side view hence why I'm trying to fix it. Plus I did a million tests and changing the shape of the cage in poop's tutorial does nothing in maya. Cody I'll sew my uv's in the future that piece of geo is just a test. I didn't actually notice any seams with the uv cuts though? I'm moving on to upresing the geo now and testing that out. It sucks that you can't just move the cage flat like in max, what a rip.
Okay it's official I fucking hate maya's normal mapper. This is the best I could come up with, adding more sides just decreases the wavyness. It still looks like shit from the side view and the whole idea with normal mapping this piece of shit is so I can use an 8 sided cylinder. I tried the 16 sided cylinder and there was still a wave. The only things that helped the problem was slanting the edges of the low res loops. So seriously is everyone hand painting their normal maps in maya this seems like a total fail for any workflow. Does xnormal fix this issue I am really not looking forward to adding yet another program to this workflow. Fucking maya.
BTW I totally wouldn't UV it like that, a cylinder like this (even with bevels) should always be one piece, I can't think of any sensible reason to split it unless you're re-using or tiling segments of it. Soften the edges and use a cage for raycasting, it will be much more optimal and you won't have to worry about hiding seams.
It's not Maya's fault, that's just how the maths work. Because you've got an eight-sided cylinder, some parts are going to be further away from the high poly than others and the rays will hit differently. So long as your lowpoly doesn't have as many sides as the high poly, you'll always get the waviness.
These shouldn't be too hard to just pull straight in Photoshop, though.
Mop: Unless im wrong and there is some secret setting somewhere, adjusting the cage in maya only adjusts ray distance, not direction. So its moot to say just adjust the cage here.
Xnormal may fix the issue, but then you'll have to deal with setting the cage up in XN, which has a really poor UI for that sort of thing.
Just be glad you aren't using max, you would need to have every other edge hard just to be able to display a simple object like that without smoothing errors .
Maya may suck in some regards, but it is by far the most accurate when it comes to rendering and displaying normals correctly.
One idea to consider may be this:
Leave the 8 sided pipe as it is, remove the midsection from it, and then create a new midsection chunk that has more sides and just intersects, and thus solves/improves your wavy problem, while still allowing the base of the pipe to be 8 sided, to fit onto any other 8 sided pipe meshes etc.
Thanks everyone for the help on this. Here is the end result straight out of the maya bake. Moving the edges of the low poly to be less right angle helped a bit and shrinking the low poly to be a closer match to the high poly also helped. As well baking at 4x4 samples fixed a couple artifact pixels in those areas. If anyone figures out a way to avoid this issue in maya in the future without having to hand edit the normal map please let me know.
Yeah going with a softer bevel there on the edges makes since, your averaged angle there is less steep than it was with your first images, so you'll get a straighter projection.
Replies
Other causes may be your smoothing groups, is it all set just set to 1 smoothing group? it may help to seperate the groups for each on of those rings.
hope that helps! btw, love your portfolio
If there is a cage you can adjust in Maya try this, it will help minimize it but probably won't eliminate it.
You can also clean it up by hand a few ways.
Use the Filter > Liquify to nudge the shapes around. If you need to clean up multiple layers (like an AO bake also) save the cage and load it onto the other layers.
You can also select a single row of pixels on the edge of the cylinder unwrap and drag it out across the waves.
edit: Ah, Vig beat me to it. *shakes fist*
The first thing here is a max specific fix, maya's cage only adjusts ray distance, not ray direction, so tweaking the cage doesnt actually change anything.
The second solution is ok, but can be a real big pain in the ass if you need to edit multiple maps (n, ao, color bake, etc). Or if you have a complex uv layout, or your uv edges arent all straight.
So its best to instead try to understand WHY this happens, and fix the problem with your geometry at the start.
So, the reason WHY you get the wavy lines is this:
When you use a lowpoly mesh to project onto a high, it uses the vertex normals for the ray direction. Since your vertex normals on those areas where you have the beveled out midsection are pointing somewhere near 45*, you're not just projecting straight out. You get the wavy lines because you're seeing the difference between the high and the low. To solve this with geometry you have pretty much two options:
1. Make the vertex normals more accurate, you can do this by cutting in more support loops. You'll be adding extra loops that arent doing much or anything to the silhouette so this may not be the best solution, but to account for wavy lines/skewed details sometimes its a must.
2. Lessen the difference between the high and the low, the more closely your low matches to your high, the less "wavy difference" you'll get in your NM. So you could try 12 sides, or 16 sides to get a better result. This method generally results in an overall better asset.
Now of course you can't always just add more geometry, but that is often the best solution to many normal projection problems, be it fixing wavy lines, skewed details, or getting rid of pesky smoothing errors.
Some people get really anal about these wavy lines, but me unless they are very bad(your example is pretty bad) i wont worry about them unless they are on a very important/highly visible part of the asset, then again, adding the right amount of geometry there not only fixes the bake errors but gives a better overall asset.
In a lot of cases, depending on how you view the object, the wavy lines can actually help to make it appear more round, from certain angles, of course it can look much worse from certain angles as well, so it helps to be mindful of how the asset will be viewed. In fact the area you have circled actually appears to be more round than it actually is, which is a plus, however the bottom is obviously wavy. Really in few cases do i ever want to eliminate it entirely, because then you are just showcasing how lowres your mesh actually is, and the goal should be to emulate the look of the high more than anything, instead of going crazy over slight wavyness.
http://boards.polycount.net/showthread.php?p=999702
These shouldn't be too hard to just pull straight in Photoshop, though.
Xnormal may fix the issue, but then you'll have to deal with setting the cage up in XN, which has a really poor UI for that sort of thing.
Just be glad you aren't using max, you would need to have every other edge hard just to be able to display a simple object like that without smoothing errors .
Maya may suck in some regards, but it is by far the most accurate when it comes to rendering and displaying normals correctly.
One idea to consider may be this:
Leave the 8 sided pipe as it is, remove the midsection from it, and then create a new midsection chunk that has more sides and just intersects, and thus solves/improves your wavy problem, while still allowing the base of the pipe to be 8 sided, to fit onto any other 8 sided pipe meshes etc.