Hello. I'm continuously trying to bake normals for about three days now, using various approaches. I've read tons of manuals, but it seems that my results even worse with every iteration. I desperately need help.
I create high-poly and low-poly meshes in Blender 2.66
Here is high-poly
Here is low-poly
They looks pretty identical, because low-poly is just a copy of high-poly before subdivision modifier was applied (4 levels of subdivision).
Here is low-poly mesh (red lines are seems for unwrapping)
Here is UV-map for low-poly
I setup a cage in xNormals and trying to generate normals, but this is what i receive
This is how Blender3D generate normal maps
Anyway the result is catastrophic, no matter what map i'm using
I have some black dots on model, and surface also strongly deformed. And whole model is a mess.
I think I've managed to crack it thanks to this thread I was having a lot of issues but reduced them significantly by not having such harsh edges in my low poly.
I'm new to this though so I'd love feedback on whether this is an acceptable amount of wavyness and if I could further optimise my mesh? I'm getting some small errors where the boxes on the right are intersecting the cylinder, any ideas how I can fix that? Would it be better to create the high poly all from one mesh?
It's getting there but you still have some pretty major waviness and skewing, which should be fixed before moving on.
Add some support loops to control the projection (fixes skewing) and increase the side count to fix the waviness.
6 sides(?) is way too low for something that is going to be close to the camera like this so I would bump it up to at least 12 sides, possibly more.
You can fix all this stuff and retain the same tri count that you have now but it requires a more complicated workflow and Object space normal maps which shouldn't really be used until you ave a slightly firmer grip on the theory of why you are getting the issues in the first place. Best to fix the cause and not the symptom
Also you need to add more bevels to you high poly mesh as 90° angles don't read well on normal maps as there is no surface change. You have the bevels on your low poly already, just add the same bevels on your high poly too.
Really it just needs more sides, even if you "fix" the waviness, eg: so its all straight, its still going to look bad because 6 sides is not enough, unless this object will be very, very small in game, in which case the waviness wouldn't be noticeable either.
Thanks for the feedback guys as always - much appreciated!
The control loops did a great job fixing some of the distortion. The model is primarily just for learning purposes but the object it's based on is a very small part of overall mesh.
I have no idea what object space normal maps are but I'll look into this when I can.
Its really, really important to keep this in mind: In a lot of cases, if you end up cutting in extra edge loops to "control" waviness, you could simply use the same amount of triangles make the object have more sides, which will look better in every situation.
Yeah matching your lowpoly to your highpoly is more important than control loops, that also means making a highpoly optimized to bake better for low poly.
Not sure if this is the best spot to put this, feel free to delete if not.
I've been trying to get my head around normal maps and I've applied it to my model, but the sphere in the model is now rendering out all the edges whenever I apply the normal map.
In B the sphere is smooth enough, the moment I add a normal map to the whole model, the edges start showing in A. I've tried removing the detail altogether by using the neutral 128,128,255 and effectively deleting the sphere off the normal map, this still results in the look of A. Why is this happening?
Perhaps someone more experienced can help you but at first glance I would say that there's something wrong with your smoothing groups on Sphere A. Try applying one smoothing group to the entire object.
"When you use object-space normal maps the vertex normal problem goes away since you're no longer relying on the crude vertex normals of the mesh. An object-space normal map completely ignores vertex normals. Object-space mapping allows you to use all soft edges and no bevels on the low-res mesh, without showing lighting errors."
So it basically means that baking with object space maps is more vertex efficient, and requires less retopo work?
Considering the fact that you can convert object space maps into Tangent space, doesn't it make OS-maps the better choice?
If I understand this correctly, then baking an OS Map, and then converting it to TS, is actually faster and more efficient than bevelin' and tangentin'.
Object space normal maps cannot be used on anything that deforms or animates, the lighting will break as soon as you move vertexes around. I also THINK (not sure on this) tangent space normal maps compress better so they take up less memory.
Assuming they're swizzled, tangent space maps compress better, but take up more memory.
Object-space maps would get compressed to DXT1 (5,6,5 bits respectively for R,G,B). However that's generally too low a precision for really good object space maps, so it's results are sucky due to compression. Even full 8bit uncompressed textures barely contain enough information for an object space map - the precision is just too low - which is why when using things like Handplane it's a good idea to store your object space maps with at least 16bit precision.
Tangent space maps get stored as DXT5 (5,6,5,8 bits respectively for R,G,B,A - but only the G and A channels are used). The 5 bit channels aren't quite enough for tangent space maps, so the R and G channels get stored in G and A, with the original B value reconstructed inside the shader*. That's generally just about fine precision for tangent space maps.
*The reason you can do this with tangent space maps and not object space maps is the Z value (blue channel) never goes into negative values. Technically you could compress object space maps in the same way and store a sign value in one or both of the disused channels (R or , but the precision would still likely be too low for a good object space map.
Tangent space maps, effectively, store up to a hemisphere of information, where object space maps must store a sphere of information (which would require double the precision for the same visual result).
Also, object/world space maps will generally break when you use batching on rotated meshes, as the object transform will get changed.
Object space normal maps cannot be used on anything that deforms or animates, the lighting will break as soon as you move vertexes around.
This is a totally incorrect but common misconception. Object space maps certainly can be animated, your shader simply has to keep track of the vertex transformations, which is a tiny bit more expensive (but you don't need to store normal or tangent data for the mesh so it evens out). Darkest of Days uesd animated OS maps, as did BF: Vietnam - they even used OS maps on characters, which debunks the other animation misconception (you can't use OS maps on soft deforming meshes).
The big reasons not to use OS maps are the fact that you can't instance or mirror nearly as easily with them (mirroring can be done via a special shader), and its much more difficult to add extra hand painted detail, so for practical workflow concerns they are a pain.
When applying my normal map to my model it applies itself onto a part that it's not supposed to be on. Anything i done wrong or missed? i can post a screenshot if you need to see it.
I'm new to baking normals and this might have been already discussed here but I can't find the discussion. So the question is - if my normal maps look correct in Marmoset and Xnormal but look like shit in UDK and Unity does it happen because I messed some import settings? Or may be Xnormal and Marmoset share the same tangent space? I used Handplane to bake the map for UDK and Unity but they still look horrible. I can't post screens right now, but may it's a common problem and someone can point me in a right direction.
I'm new to baking normals and this might have been already discussed here but I can't find the discussion. So the question is - if my normal maps look correct in Marmoset and Xnormal but look like shit in UDK and Unity does it happen because I messed some import settings? Or may be Xnormal and Marmoset share the same tangent space? I used Handplane to bake the map for UDK and Unity but they still look horrible. I can't post screens right now, but may it's a common problem and someone can point me in a right direction.
Xnormal and Toolbag's default normal map display is pretty similar, both are pretty similar to Unity too, so try loading the map you have in Marmoset directly into unity (and make sure its tagged as a normal map in Unity). If that looks ok, your settings in Handplane are probably the issue.
So I did three tests and they all look horrible in Unity but ok in apps they were baked in.
From Xnormal to Unity:
From Substance Designer to Unity (SD has the same tangent space so it should be fine in theory):
From Handplane to Unity:
Here are high and low poly meshes + all test maps. I did the most basic bake without smoothing groups, UV splits and cage. Just wanted to try it and see if something wrong with workflow before trying to improve on it. The mesh was triangulated before baking.
finally won this little technical fight. I had to turn off "create from grayscale" option. Now it looks alright. I also did the second bake with cage and UVs/edges split. It looks a bit cleaner
Hi, i was optimizing my normalmap-baking workflow and i have an issue.
I was baking a simple smoothed cube and i always get artifacts at the seams and on all flat surfaces.
- I am baking in xNormal with an external cage.
- The cage/averaged projection mesh was done in maya, via transfer maps -> envelope -> average normals.
- My lowpoly normals are all smooth, except for the uv-border edges, those are hard.
-I baked an Object Space Nomal Map, brought it into handplane and generated my Maya-synced Tangent Space Normal Map.
My first try was with a cube, consisting out of six flat planes. The shading was completely messed up, so i beveled the edges. I checked the highpoly, the flat sides are all smooth, no errors on that side.
What am i doing wrong? Is my lowpoly the problem? Or my averaged projection mesh?
Neither of the issues you've got here are what you think they are.
Both come down to bit-depth and resolution. I think you'll find that if you increase the resolution, the "seams" will be less apparent, and if you bake at 16bit per channel vs 8 bit per channel, the "smoothing errors" will go away as well. Though if you bake in Maya you'll get a little more accurate normals, you almost certainly still get some stair-stepping from bit precision issues.
These are both basic limitations of normal maps.
The good news is, once you texture this asset and put it into game, you won't notice these problems anymore. Texture compression will be much worse than the issues you're looking at here.
If you want to completely get rid of this though, use a 6 sided cube, hard edges on all sides, and separate uv islands for each face.
You were right about the seams and resolution. But even with a 4k normalmap i cannot get rid of them completely. Is it not possible to have an entirely flawless bake without any seams on close distance?
About the 'smoothing errors' a 16-bit map didn't change those. I'm not sure bit precision is the cause of the problem. Take a look at the shot below. It does look like my highpoly is messed up, because the artifacts are not typical 'stair-stepping-effects'. When i bake a 6-sided cube however, all uvs separated, i get a nearly perfect flat surface on these areas -.-
maybe i'm a little too precise here and it would not matter ingame anyways, but i just want to understand all of this entirely...
EDIT: btw, it doesn't matter if i bake in maya or xnormal, it always looks the same
Hey, has anyone here played Bioshock Burial at Sea episode 2? Did anyone else notice how Booker's guitar in the very beginning has very apparent waviness in its normal map at the edge facing the light source? It's funny to see an issue that is being humbly tackled in a technical forum out of the way appearing fairly prominently in a AAA production. Although to be fair, I doubt most of the game's audience notices that waviness given that 3d artists are a minority.
(sorry, I don't have a screenshot handy right now.)
You were right about the seams and resolution. But even with a 4k normalmap i cannot get rid of them completely. Is it not possible to have an entirely flawless bake without any seams on close distance?
About the 'smoothing errors' a 16-bit map didn't change those. I'm not sure bit precision is the cause of the problem. Take a look at the shot below. It does look like my highpoly is messed up, because the artifacts are not typical 'stair-stepping-effects'. When i bake a 6-sided cube however, all uvs separated, i get a nearly perfect flat surface on these areas -.-
maybe i'm a little too precise here and it would not matter ingame anyways, but i just want to understand all of this entirely...
EDIT: btw, it doesn't matter if i bake in maya or xnormal, it always looks the same
I see very clear stair stepping/bit depth issues here. Bake tiff in XN(which should be 16bit by default) and then convert back to 8bit, which should dither. You can also bake 2x in maya, open in PS, set to 16 bit, size down, convert to 8bit, I've had good luck with this method, it tends to dither effectively as well.
Earthquake is perfectly correct. This is an issue with the texture not having enough precision to accurately produce a very subtle gradient. When you have a flat/nearly flat surface and you're trying to use a tangent space that has a very low curvature, it becomes problematic and actually makes it even more difficult to render the normal map than it would have been with all smooth normals. This is because the normals are transformed into a space that bends ever so slightly and everything gets dropped to 8 bit. As such, when you transform back out in the pixel shader it's broken.
Even when you add hard edges around the UVs you are still going to get a small amount of gradation due to the parts that are connected in UV space and the subtle transition between hard/soft normals. You don't get the same issues when you have 6 UV islands because there is no need to have the same subtle transition between hard/soft normals as they are all hard.
Another issue is that in many cases the flat surfaces on the high poly are not actually 100% flat if you have scaled or transformed them. Even changing the triangulation on a mesh can cause it to change size by a very small amount. You can also get some sort of low level curvature due to the way that CC subd works too even though the surface looks flat i.e. the top, bottom and side edges are ever so slightly smaller than the centre of the cube.
The minor seams you are getting are not beyond what I would consider normal and are most likely due to texture filtering or something similar.
I would bet that the exact same errors are visible on the normal map if you add a levels and clamp the values a little. Try it in PS and see
I've tried now 2 hours to figure out this problem. I don't understand what's wrong.
This is the upper side of the wing. The checkered one is the low poly and the grey one is the high poly.
This is just a single poly.
Can anyone explain me the behaviour of the normal map? What have I done wrong?
Ok, couple things you can try:
1. Add more geometry to your low so it more closely matches the high, add bevels etc, as its really, really, really low poly right now
2. Add hard edges on the uv borders, this sort of smoothing is too difficult for a normal map to account for
I managed go get almost rid of this issue by just adding some support loops to the side of the wings to define the edges of the low poly. This is kind of strange to me, cause it's not altering the shape at all and as far as I have understood normal map baking it's about fireing raycasts between the high poly (cage) and the low poly, so as long as the shape is 100% identical, the result should be the same.
I'm gonna post some new shots later today and I'll also try some different baking processes. I really want to find out what went wrong.
@Quack!
I was baking this in Substance Designer and the objects have been exported as .OBJ from Max. I think I'll also try to do a bake directly in Max and see if there is any difference later today or try some different file formats.
And just for the information: I've tried also to bake a world space normal map and that seems to look fine.
The heavy gradiation in your normal map suggest that you haven't set your edges to hard at the uv borders, or perhaps your mesh normals are not being exported to substance.
The heavy gradiation in your normal map suggest that you haven't set your edges to soft at the uv borders, or perhaps your mesh normals are not being exported to substance.
Okay now I'm confused. When I understood the other threads correctly, you guys were propagating, that on UV borders you can have "free" hard edges without any issues. So why should I soften them?
Even if there are hard edges. On the same model I have a lot of similar situations and all of them are fine in the normal map.
Is there a way to check that the mesh normals are exported correctly?
I thought that edges that have an angle of 90° or acute are considered hard. I'm pretty sure, I have a 90° angle on all edges except maybe the one that connects the wing to the body, which is probably 100°.
Did I misunderstood something regarding hard edges?
Hard edges have little to do with the angle of the geometry. A hard edge is something you define, in max by using smoothing groups and in Maya by tagging the edges as hard. Though there are some automated ways to define hard edges by angle in both apps.
Replies
rendering>gamma/lut setup, disable.
I create high-poly and low-poly meshes in Blender 2.66
Here is high-poly
Here is low-poly
They looks pretty identical, because low-poly is just a copy of high-poly before subdivision modifier was applied (4 levels of subdivision).
Here is low-poly mesh (red lines are seems for unwrapping)
Here is UV-map for low-poly
I setup a cage in xNormals and trying to generate normals, but this is what i receive
This is how Blender3D generate normal maps
Anyway the result is catastrophic, no matter what map i'm using
I have some black dots on model, and surface also strongly deformed. And whole model is a mess.
I'm new to this though so I'd love feedback on whether this is an acceptable amount of wavyness and if I could further optimise my mesh? I'm getting some small errors where the boxes on the right are intersecting the cylinder, any ideas how I can fix that? Would it be better to create the high poly all from one mesh?
(Baking in 3DS Max - displaying in Marmoset)
Add some support loops to control the projection (fixes skewing) and increase the side count to fix the waviness.
6 sides(?) is way too low for something that is going to be close to the camera like this so I would bump it up to at least 12 sides, possibly more.
You can fix all this stuff and retain the same tri count that you have now but it requires a more complicated workflow and Object space normal maps which shouldn't really be used until you ave a slightly firmer grip on the theory of why you are getting the issues in the first place. Best to fix the cause and not the symptom
Also you need to add more bevels to you high poly mesh as 90° angles don't read well on normal maps as there is no surface change. You have the bevels on your low poly already, just add the same bevels on your high poly too.
The control loops did a great job fixing some of the distortion. The model is primarily just for learning purposes but the object it's based on is a very small part of overall mesh.
I have no idea what object space normal maps are but I'll look into this when I can.
Not sure if this is the best spot to put this, feel free to delete if not.
I've been trying to get my head around normal maps and I've applied it to my model, but the sphere in the model is now rendering out all the edges whenever I apply the normal map.
In B the sphere is smooth enough, the moment I add a normal map to the whole model, the edges start showing in A. I've tried removing the detail altogether by using the neutral 128,128,255 and effectively deleting the sphere off the normal map, this still results in the look of A. Why is this happening?
"When you use object-space normal maps the vertex normal problem goes away since you're no longer relying on the crude vertex normals of the mesh. An object-space normal map completely ignores vertex normals. Object-space mapping allows you to use all soft edges and no bevels on the low-res mesh, without showing lighting errors."
So it basically means that baking with object space maps is more vertex efficient, and requires less retopo work?
Considering the fact that you can convert object space maps into Tangent space, doesn't it make OS-maps the better choice?
If I understand this correctly, then baking an OS Map, and then converting it to TS, is actually faster and more efficient than bevelin' and tangentin'.
Object-space maps would get compressed to DXT1 (5,6,5 bits respectively for R,G,B). However that's generally too low a precision for really good object space maps, so it's results are sucky due to compression. Even full 8bit uncompressed textures barely contain enough information for an object space map - the precision is just too low - which is why when using things like Handplane it's a good idea to store your object space maps with at least 16bit precision.
Tangent space maps get stored as DXT5 (5,6,5,8 bits respectively for R,G,B,A - but only the G and A channels are used). The 5 bit channels aren't quite enough for tangent space maps, so the R and G channels get stored in G and A, with the original B value reconstructed inside the shader*. That's generally just about fine precision for tangent space maps.
*The reason you can do this with tangent space maps and not object space maps is the Z value (blue channel) never goes into negative values. Technically you could compress object space maps in the same way and store a sign value in one or both of the disused channels (R or
Tangent space maps, effectively, store up to a hemisphere of information, where object space maps must store a sphere of information (which would require double the precision for the same visual result).
Also, object/world space maps will generally break when you use batching on rotated meshes, as the object transform will get changed.
This is a totally incorrect but common misconception. Object space maps certainly can be animated, your shader simply has to keep track of the vertex transformations, which is a tiny bit more expensive (but you don't need to store normal or tangent data for the mesh so it evens out). Darkest of Days uesd animated OS maps, as did BF: Vietnam - they even used OS maps on characters, which debunks the other animation misconception (you can't use OS maps on soft deforming meshes).
The big reasons not to use OS maps are the fact that you can't instance or mirror nearly as easily with them (mirroring can be done via a special shader), and its much more difficult to add extra hand painted detail, so for practical workflow concerns they are a pain.
Most engines will not support it out of the box, you need some custom shader work to get it running.
Xnormal and Toolbag's default normal map display is pretty similar, both are pretty similar to Unity too, so try loading the map you have in Marmoset directly into unity (and make sure its tagged as a normal map in Unity). If that looks ok, your settings in Handplane are probably the issue.
From Xnormal to Unity:
From Substance Designer to Unity (SD has the same tangent space so it should be fine in theory):
From Handplane to Unity:
Here are high and low poly meshes + all test maps. I did the most basic bake without smoothing groups, UV splits and cage. Just wanted to try it and see if something wrong with workflow before trying to improve on it. The mesh was triangulated before baking.
I was baking a simple smoothed cube and i always get artifacts at the seams and on all flat surfaces.
- I am baking in xNormal with an external cage.
- The cage/averaged projection mesh was done in maya, via transfer maps -> envelope -> average normals.
- My lowpoly normals are all smooth, except for the uv-border edges, those are hard.
-I baked an Object Space Nomal Map, brought it into handplane and generated my Maya-synced Tangent Space Normal Map.
My first try was with a cube, consisting out of six flat planes. The shading was completely messed up, so i beveled the edges. I checked the highpoly, the flat sides are all smooth, no errors on that side.
What am i doing wrong? Is my lowpoly the problem? Or my averaged projection mesh?
Both come down to bit-depth and resolution. I think you'll find that if you increase the resolution, the "seams" will be less apparent, and if you bake at 16bit per channel vs 8 bit per channel, the "smoothing errors" will go away as well. Though if you bake in Maya you'll get a little more accurate normals, you almost certainly still get some stair-stepping from bit precision issues.
These are both basic limitations of normal maps.
The good news is, once you texture this asset and put it into game, you won't notice these problems anymore. Texture compression will be much worse than the issues you're looking at here.
If you want to completely get rid of this though, use a 6 sided cube, hard edges on all sides, and separate uv islands for each face.
About the 'smoothing errors' a 16-bit map didn't change those. I'm not sure bit precision is the cause of the problem. Take a look at the shot below. It does look like my highpoly is messed up, because the artifacts are not typical 'stair-stepping-effects'. When i bake a 6-sided cube however, all uvs separated, i get a nearly perfect flat surface on these areas -.-
maybe i'm a little too precise here and it would not matter ingame anyways, but i just want to understand all of this entirely...
EDIT: btw, it doesn't matter if i bake in maya or xnormal, it always looks the same
and yes, you're too precise on this, we have limitation, that's why we use normal map instead of a "hi-poly"
(sorry, I don't have a screenshot handy right now.)
I see very clear stair stepping/bit depth issues here. Bake tiff in XN(which should be 16bit by default) and then convert back to 8bit, which should dither. You can also bake 2x in maya, open in PS, set to 16 bit, size down, convert to 8bit, I've had good luck with this method, it tends to dither effectively as well.
Even when you add hard edges around the UVs you are still going to get a small amount of gradation due to the parts that are connected in UV space and the subtle transition between hard/soft normals. You don't get the same issues when you have 6 UV islands because there is no need to have the same subtle transition between hard/soft normals as they are all hard.
Another issue is that in many cases the flat surfaces on the high poly are not actually 100% flat if you have scaled or transformed them. Even changing the triangulation on a mesh can cause it to change size by a very small amount. You can also get some sort of low level curvature due to the way that CC subd works too even though the surface looks flat i.e. the top, bottom and side edges are ever so slightly smaller than the centre of the cube.
The minor seams you are getting are not beyond what I would consider normal and are most likely due to texture filtering or something similar.
I would bet that the exact same errors are visible on the normal map if you add a levels and clamp the values a little. Try it in PS and see
as it seems you are both right and i was an idiot :poly124:
Guess I will bake in 16bit from now on...
This is the upper side of the wing. The checkered one is the low poly and the grey one is the high poly.
This is just a single poly.
Can anyone explain me the behaviour of the normal map? What have I done wrong?
Thanks!
What does the high look like?
What does the low look like (wires etc)?
What do the uvs look like?
Low-Poly
Wire Highpoly:
1. Add more geometry to your low so it more closely matches the high, add bevels etc, as its really, really, really low poly right now
2. Add hard edges on the uv borders, this sort of smoothing is too difficult for a normal map to account for
Edit:
What do you mean with "2. Add hard edges on the uv borders, this sort of smoothing is too difficult for a normal map to account for" exactly?
I have hard edges everywhere on the mesh and the distorion is on blank side, not near any edges.
Can you post another shot of your lowpoly with wires? It might be the point where the wing connects to the body that is causing the remaining issues.
I have had errors that were this weird looking and random before and it was a bad model format.
I'm gonna post some new shots later today and I'll also try some different baking processes. I really want to find out what went wrong.
@Quack!
I was baking this in Substance Designer and the objects have been exported as .OBJ from Max. I think I'll also try to do a bake directly in Max and see if there is any difference later today or try some different file formats.
And just for the information: I've tried also to bake a world space normal map and that seems to look fine.
Just added some support loops on side of the wing.
This results in this normal map:
There are still some issues on the right side, however they are barely viewable.
Still wondering why issue occurs. I will do some more research.
By the way, I get the same result, when I export the meshes as FBX.
Yes I totally agree that the issue just moved, but I don't know what causes this issue. I'm using Max 2015, but I'm pretty sure it's not a max issue.
Okay now I'm confused. When I understood the other threads correctly, you guys were propagating, that on UV borders you can have "free" hard edges without any issues. So why should I soften them?
Even if there are hard edges. On the same model I have a lot of similar situations and all of them are fine in the normal map.
Is there a way to check that the mesh normals are exported correctly?
Did I misunderstood something regarding hard edges?