Normal Map Banding
polycounter lvl 11
So I I have this normal map I baked in xnormal, also tried in modo, and got the same result. It has banding pretty bad, I'm wondering if anyone knows any tips or tricks to remove it.
Here is a link to the normal map (you can see the same banding pattern if you adjust the levels): http://dl.dropbox.com/u/2997790/xnormalNormalMap_normals.tga
Here is the low in game (looks the same in xnormal):
Here is the high:
Here is a link to the normal map (you can see the same banding pattern if you adjust the levels): http://dl.dropbox.com/u/2997790/xnormalNormalMap_normals.tga
Here is the low in game (looks the same in xnormal):
Here is the high:
Replies
http://boards.polycount.net/showthread.php?p=1034472#post1034472
I have no idea if there is a way to fix it though.
Like I said you can see this in the normal map if you adjust the levels. So its actually in the image. I tried baking the model from xnormal, 3dsmax, and modo and got the same results across the board.
EDIT:
Here are the OBJs i'm using for the xnormal bake in case anyone wants to give it a go: http://dl.dropbox.com/u/2997790/bandingIssue.rar
EDIT: Here are the OBJs I am using in case anyone wants to have a look: http://dl.dropbox.com/u/2997790/bandingIssue.rar
I think it may just be something I can't fix. The baking software might be able to take this into account though but I don't know what I have any control over it...
Editing the lwoply model .obj I can see strange things coming from that Modo exporter ( like unnormalized normals, for example ( vn 0.0051 1.0000 -0.0040 ), so I bet the data could be a bit malformed.
If you render a "Wireframe and ray fails", you can also see there are some CW-defined UVs which can be problematic ( curiously the top of the wing is ok ). The UV faces should be defined CCW ( the vertex faces too ).
I think some old versions of Photoshop had problems with the TGA format. Try with other format lilke a .BMP to be sure.
The Photoshop levels destroy completely the normal map normalization, so it's perfectly normal to see strange things for a normal map with a relatively low number of "colors".
You can try also to set the lowpoly normals mode from "use exported normals" to "average vertex normals" and see what happens.
Btw, what xNormal version are you using?
He wrote the perl script I used to export the objs from modo: http://forums.luxology.com/discussion/topic.aspx?id=41509
That said, I imported those objs into maya and they looked fine. I re-exported them from there and got the exact same banding issue.
Keep in mind I also get the same banding issues when I bake in Modo where I created the mesh. So the banding doesn't appear to be related to the objs?
Please correct me if I'm wrong but isn't this banding occurring simply because there is shading that can't accurately be captured in a 32bit image, and really, unless its somehow mathematically compensated for, there isn't a way to fix this?
Anyways, here is a bake I did with the NEW objs that I exported out of Maya.
You can still see the banding happening there, not as easy to see, just look close, sorry I was in a hurry and just took the first pic where you could see it.
Here are the OBJs I used: http://dl.dropbox.com/u/2997790/fromMayaThisTime.rar
I know adjusting the normals messes up the normal map. I don't actually do that when I use the texture. The texture I used that produced the banding was an unmodified texture taken directly from xnormal.
I only did that to show the banding pattern could be seen in the image and it looked exactly the same as it did when it showed up in engine.
I get the exact same banding effect with PNG, BMP, DDS, and assuming all the rest
I rendered it clean out of xsi. no problem. It also renders fine.
also, the tessellation on both those things is whack.
While I can't see the banding in your XSI normal map I do see a ton of other artifacts. Are these caused by the jpg compression? Could you upload a bmp so I can check and see if the banding is actually not there, the artifacts are so bad its hard to tell if its actually gone or just covered up.
I am at home now and rendered out another texture on my home computer with a newer version of xnormal. I also have the latest drivers installed on my home machine. The problem is the same.
EDIT: I just saw jogshy bit about baking with "average normals" on, I just did that and the banding still appeared.
Because it's not the normals rendered, it's something else about his computer, or a setting or something that's messed up imo.
Here's a question, have you exported the object, and brought it back into the program? Sometimes the object itself is corrupted and will do weird render stuff.
Have you tried baking a different object and seeing if the same thing happens?
Have you tried using a flat normal map ( or any other normal map you know works) on the object and seeing how it renders?
etc. etc.
The tessellation of that object might be wonky for xnormal or whatever. Anyways, you need to deduce what isn't causing the problem, instead of baking over and over.
oh btw, my normal render shouldn't do that either (Screen shot above). There's nothing wrong with that texture.
Interesting idea, but it didn't work from what I can tell, looks a little worse?
Also, to rule out this being the model. I made a new high and low model, in Maya, from scratch, and then baked it in maya and then loaded it up in xnormal.
Banding is still there.
Also, keep in mind, this doesn't show up unless you have a darker texture and a higher specular on your model. Unfortunately that's exactly what this project calls for.
Either way, put noise in the normal, or build a new lowpoly that isn't so similar to the high poly, so the normal map has more, uh.. texture .
*edit*
I looked at it in xnormal instead of marmo and I'm getting different results altogether.
I'm thinking the bake is just too much like the original object or too close or something.
Just a suggestion though that low poly geo is not good, too many faces for one , and the flow is just strange i recommend building a different low poly one that is more efficient, but just change the envelope when you bake and your problem should go away. At least it did in all my testing.
btw if you set your high poly to a dark blinn and your low poly to pure black and put them on top of each other you will see the exact places its intersecting, and those happen to correspond to the exact place that the errors where occurring and in the exact shape.
Your also right im not 100% sure its banding it just looked an awfull lot like it to me, sorry if I was wrong on that.
EDIT: I tried making another model and retopoed it, and also made sure there was no intersecting, and I got the same issue. Could you post your texture so I could make sure its not something wierd on my end.
here you go, works fine for me, on this side. its tga so that will eliminate any chance of artifacts from compression.
also might be worth noting i softened all normals, on the low poly.
^ this
The Low poly is just a SINGLE POLYGON. It sits below the highpoly a good distance. This should rule out any intersecting issues.
Also, I tested hijacks texture and the artifacts that show up there are identical to the ones I have been seeing, in fact they issue has shown up in EVERY single texture I have received save the one from XSI (from moof) which had massive compression artifacts because it was saved as a jpg.
Keep In Mind: This issue is only apparent if you have a Black texture and a high spec. In Xnormal I do the following to see it:
- Light Intensity 1.27
- Lock Light To Camera
- Move the Camera above the curved surface (top of the wing) and then move the angle around a bit.
- Make sure you have a black texture applied.
I understand that this issue probably isn't normally an issue, but this is for a model that will be very black, against a blue sky, and will have panning shots done close up of its surface. Meaning this issue WILL show up like this.I'm really curious if anyone can offer any idea as to what causes this. I have always noticed these types of issues, but usually once the diffuse is applied they go away, but this time I can't get away from it. I'm just going to have to paint it out if it comes down to it, but my technical curiosity really wants to know how to resolve this so I don't have to do that in the future.
Link to the new files: http://dl.dropbox.com/u/2997790/highLowNoIntersection.zip
If I remember well there were two major bugs in that version:
1. I used a $%%&&&"! adaptative color AA. Try to set the threshold to 0/0/0/0, but if I remember well there was also a bug in the final filtering process that was making all very blurry.
2. I compressed the normal maps to DXT3/5. That was an error because the color quantization can produce banding.
3. There was a bug computing the vertex normals.
So better update to a modern version of xNormal !
Btw, the banding can be caused also by texture compression. Sure it's disabled in your 3D engine/whatever you use to visualize the 3D models ( including the 3D viewer in xNormal via Plugin Manager ).
I checked and made sure image compression was off on both DX10 & DX9, I tried both of them and the banding persisted.
Also, over the years I have noticed banding in general on all 3D models, it just appears to be a product of using RGB colors with Gouraud Shading
Is banding in real time rendering just an excepted thing, as far as I can tell its everywhere and no one seams to really care about it, but it seams like its being amplified when its baked into my normal map for some reason.
Here are some examples of what I mean:
With DX9 is possible that your SM2.0 graphics card could expose partial FP precision, but with DX10 full 32bits FP precision is required for a WQHL driver.
Btw... have you enabled the Gamma Correction for the DX10 graphics driver in xNormal? Play with that option, it might solve the problem.
The TGA format can use only 8 bits per channel. Perhaps if you use a 16bits TIFF or OpenEXR the banding could dissapear.
Btw, do you use a 24/32 bits of color video mode or a 16 bits one? Do you use a Mac laptop?
I tried messing around the dx10 gamma correction, it didn't change anything.
Keep in mind, that the banding I see in the normal maps is the exact same pattern I see when I adjust the level photoshop (which I only do to easily see the banding, I don't ever apply that adjustment).
As far as I know any computer I have ever used has banding that shows up on any type of realtime lighting. Its usually so subtle that no one ever notices.
However, in this case the banding gets transferred into the normal map and really stands out.
This exact same pattern occurs in every normal map I have seen, all bakers including,
Modo, Maya, 3DSmax, & XSI produce the exact same banding pattern, xnormal to.
It doesn't appear to be a problem specific to my machine, or any one piece of software. You should be able to re-produce it on your end if you load up the files I provided, apply a completely black texture, and turn up the light intensity in xnormal.
I guess the point I'm trying to convey is that this issue appears to be a universal/acceptable limitations of baking normal maps, or is unknown and hardly ever noticed since it only shows up under certain circumstances.
I'm just trying to figure out which of the two it is, and if there is anything I can do to deal with it.
You have a limited number of colors you can use in an RGB image.
I'm trying to store a really subtle changes using that limited number of colors, and there simply are not enough of them to accurately describe the curvature.
Hence, I am getting banding: the renderer does its best to cram all the information into the limited range but it gets piled up like train wrecks into bands rather than a seamless smooth transition.
Also Moof, to your credit I think you also pointed out the exact same thing, I just wasn't able to wrap my head around it at that point in time I don't think.
So now my question is: Is there anything that I can do about it?
I came up with the following:
- I could just paint it out of the normal map, but that's time consuming and might not be an option if I have other details on the surface that make that difficult.
- I'm also wondering if there is some mathematical way the baker could account for this and just skip surfaces with changes this subtle.
- Is this perhaps why almost every surface in unreal 3 has a noisy normal detail map assigned to it?, I'm wondering if it helps hide banding...
If anyone has any advice, or has dealt with this type of thing before I would love to hear it. I'm just trying to understand how this issue is tackled. Is it just ignored, do people paint this out, is it something no one has ever noticed?i must have sent the wrong file, because i garuntee that the bake did with the expanded envelope had no artifacts at all whatsoever, even ran it on a black blinn, high spec etc etc. I keep getting the feeling there is something up with your hardware, or software versions, because i was only able to recreate teh problem with intesecting geometry, after adjusting envelope problem was gone, even tried checking levels in photoshop to see if it would come back up and nope.
oh and as a side note you will never ever see that once you have a diffuse and spec texture on it, little normal map errors are ussually not noticeable it he fianl product.
I could just delete the parts of the high poly that are so similar to the low poly but it just seams so messy having to deal with it that way.
But I do see your point, really normal maps are to denote more drastic difference in surface curvature than I am trying to use them for here. I guess I'm just trying to figure out if there are any work flow things I can do to eliminate this problem whole sale or if I'm gonna have to deal with it on a case by case basis.
Hijack: You might not have adjusted the levels correctly. Here is a pic of your image in photoshop:
Also I know that these errors hardly every show up but this model will be black plastic (has a high spec) and has no other texture. So its pretty much a worse case scenario for something like this showing up, and not using a normal map for it is out since I need to bake lots of floating geometry into the surface to denote seams, ports, screws, and other stuff in the surface.
Im using TARGA for everything, no compression at all. Moof uploaded an image from xsi that was a jpg, and yeah it had issues.
I think we already established this is a banding issue caused by the limited RGB range not being accurate enough to capture the very subtle difference between my high and low poly model.
My big question now is, is there any way to work around this?
It's either that, or change up either the lowpoly or high poly.
Just for reference, when I do really lightly detailed normal maps for objects like this, I just unwrapped the lowpoly, and using the uvs I put high poly detail bits (like bolts and seams and stuff) on top of a polyplane with the uvs as a texture on it on it, and just baked that when I got it all laid out.
imagine if you will, your texture with the uv coords under all that stuff.
that's one way to avoid some issues with stuff like this. You can always do a blend of all sorts of techniques to get what you need, and that can include just going in and manually painting out stuff.
I think now I understand:
- Why the banding is occurring,
- That there is nothing I can do about it directly
- The only way around it is to modify my work flow a bit or paint it out.
I really appreciate everyones help with this, understanding these issues from all angles is really helpful to me, maybe it helped someone else to.EDIT: The one thing I am wondering though is how this sort of thing is handled in non-realtime renders like those for film. They seam to have ways of working around this issue.
AFAIK they use higher bit depth textures when needed, like 16bpc displacement maps instead of 8pc (256 grays). They're not as concerned about memory as we are!
ive had these since day one normal mapping, thats why all the vehicles that are really shiney and flat surfaced dont use normal mapping from high to low...IME
there is a limit to the fidelity in the tangent offset...and if you have alot of polys in your low, its often worse to have a normal map as your stretching a few values across a large space ontop of speed and dds artefacts its another reason to not use normal maps at certain times
http://kdc.ethernia.net/wp-content/uploads/tut2.jpg