Home› Technical Talk

Annoying AO error

3Dyan
3Dyan
polycounter lvl 13
Offline / Send Message
3Dyan polycounter lvl 13
Hi there.

At times when I'm baking AO, areas which have intersecting faces are not being occluded fully. Example:

AOerror.jpg

I'm baking with 3DSMax, and the areas of the mesh that are hidden act as if they are not occluded at all (expected) however they bleed out a bit and this is what's causing these white edges.

What is the best way to overcome this?
· Share on FacebookShare on Twitter

Replies

  • Mark Dygert
    Options
    Offline / Send Message
    Mark Dygert mod
    1) What are you baking With?
    MentalRay AO, Scanline shadowmap, Scanline lightmap, scanline complete map?


    2) Does it look different if you render it?


    3) Are you baking with enough padding?
    If your viewport resolution set to the default or lower and so it's displaying a lower level mip-map, allowing the white space to bleed in. The same thing will happen in most game engines and you should bake with more padding.


    4) How is your viewport setup? Direct3D, OpenGL, Nitrous?
    If you're using Direct3D make sure you follow step 2 of this tutorial.
    If you're using nitrous use the Nitrous Texture Tweaker to increase the resolution in the viewport.


    5) What does that section of the AO texture look like?
    · Share on FacebookShare on Twitter
  • 3Dyan
    Options
    Offline / Send Message
    3Dyan polycounter lvl 13
    This happens in MR, scanline lightmap and xnormal.
    This isn't a padding issue, as these meshes intersect and continue a little into each other.
    This is visible in every software I use (max, toolbag, udk, etc...).

    Here is what the map looks like:
    BadAOmap.jpg
    It is as if the occlusion ends a pixel or two prematurely.
    · Share on FacebookShare on Twitter
  • Bryan Cavett
    Options
    Offline / Send Message
    Bryan Cavett polycounter lvl 19
    Try making your material 2-sided on the object and then baking the ao/lightmap. I cant remember if that trick is for mental ray or lightracer though...
    · Share on FacebookShare on Twitter
  • Valandar
    Options
    Offline / Send Message
    Valandar polycounter lvl 18
    In Max, have you tried assigning a 2-sided material to the objects before baking? I used to have that issue before I did that, and really couldn't for the life of me explain why the 2-sided material works, but it did, in both scanline and Mental ray.

    Edit: Ninja'd by Bryan!!!
    · Share on FacebookShare on Twitter
  • 3Dyan
    Options
    Offline / Send Message
    3Dyan polycounter lvl 13
    I tried that. The result is the same, only now I'm getting abrupt black edges instead of white.
    Not pretty.
    · Share on FacebookShare on Twitter
  • Bryan Cavett
    Options
    Offline / Send Message
    Bryan Cavett polycounter lvl 19
    I think I've seen this happen with someone at work. They were trying to project matIDs to matching matIDs and mental ray didn't anti-alias the intersection point or changing matID edge (if that makes sense). I don't think we ever found a solution.
    · Share on FacebookShare on Twitter
  • Noors
    Options
    Offline / Send Message
    Noors greentooth
    I think if you boost the sampling quality you'll get more AA. But if the sampled texel is "inside" the mesh, i'm affraid you'll alway get a bit of bleeding on intersecting geo. We'll need some kind of offset. What came to my sick mind is to put a little negative push modifier on top of the model for baking, so the dark area start 1 or 2 pixel further.
    test :
    Ao_titties.png
    Ao_titties.pnghttp:
    · Share on FacebookShare on Twitter
  • 3Dyan
    Options
    Offline / Send Message
    3Dyan polycounter lvl 13
    That's an interesting idea, I just might try that on my next asset.
    Good thinking.
    · Share on FacebookShare on Twitter
Sign In or Register to comment.