Home Technical Talk

borders on rendered transparancy

shardy
polycounter lvl 18
Offline / Send Message
shardy polycounter lvl 18
Im having some issues with borders on rendered transparency I am using 3dsmax.

I am rendering out some 3d into sprites, they are rendered with a transparent bg however its apparent that the frames all have slight bordering around the actual 3d model of the 3dsmax environment background color (pink in this case).

Is there something I can do to avoid this happening?

Thanks

Replies

  • kodde
    Options
    Offline / Send Message
    kodde polycounter lvl 19
    Pre-multiplied alpha, try turning that off in render settings. It ought to remove what you are describing, but you might get slightly more "jagged" edges.

    Will your sprites be used against a solid background color always or varying colors? If it's a constant solid color, just match that in 3DSMAXs background color in the renders and you don't have to bother with this issue at all.
  • shardy
    Options
    Offline / Send Message
    shardy polycounter lvl 18
    Thanks for the tip, background wont be a constant solid so cant do that :(

    Will give it a crack !
  • kodde
    Options
    Offline / Send Message
    kodde polycounter lvl 19
    Actually, it doesn't have to be "constant solid", if it's generally one color all over (dark for example) you can get away with this. You will actually get more smooth transparency if you can keep this effect and use it to your advantage.

    Either way, try finding this feature, render with and without to see the effect.

    Good luck.
  • shardy
    Options
    Offline / Send Message
    shardy polycounter lvl 18
    forgive my ignorance, but where can i find it to turn it off :)
  • pior
    Options
    Offline / Send Message
    pior grand marshal polycounter
    dont bother with saving the transparency as an alpha channel

    In case you are applying a texture with transparency :
    - Make your diffuse a flat bitmap (RGB), and make the surrounding color around the sprite fairly dark
    - Create a separate blanc and white image for transparency, also without alpha channel. Load that one in the transparency slot.

    In case you are rendering something and want to make it a sprite :
    - render against a dark background,
    - render another black and white pass for he masking, and keep that pass separate for compositing

    Easy, and impossible to fuck up!
  • shardy
    Options
    Offline / Send Message
    shardy polycounter lvl 18
    Wouldn't that mean id have to manually apply the b/w pass as the alpha in photoshop for each sprite?
  • kodde
    Options
    Offline / Send Message
    kodde polycounter lvl 19
    pior> In your second scenario wouldn't you want to turn off the premultiplied alpha effect either way as to not get the background mixed with the border pixels? At least in your color pass.

    EDIT. Unless you could possibly turn off Alpha all together and having the rendered therefor not considering premultiplying at all I guess. Not familiar with Max.
  • pior
    Options
    Offline / Send Message
    pior grand marshal polycounter
    wait- maybe we are talking about different things.
    Could you post a picture of what you are trying to achieve. For instance, you could be trying to do sprites with 'solid' masking, meaning that a pixel is fully transparent or is not, but you end up fighting against the antialiasing of your renders?

    Please post pictres of :
    -your current result, broken between RGB and A passes
    -an example of the type of sprite transparency you are after.

    Kodde : The reason why I always separate RGB from A in two images in every transparency work I have to do (either compositing, or applying transparency to a texture) is to avoid exactly this kind of questions :P I don't want to battle against file formats or terminology, so I just screw that and work the easy way.

    Not sure what you mean with background mixed with border pixels. When rendering sprite from 3D you dont want any antialiasing on the edges anyways so why would you get any color mixing? You lost me here.
  • shardy
    Options
    Offline / Send Message
    shardy polycounter lvl 18
    hrm i'm not surprised, i'm not super clear with my explanation ;)

    pinkedges.png


    I have cropped and zoomed in a bit into a portion of the image, in 3dsmax i have my environment bg set to pink (255, 0, 255) and have rendered out the character as PNG with alpha, as you can see the character has a border the same color as the background.

    What I am after is nothing surrounding the 3d model, just the model itself rendered out with what I assume would be a hard transparency? (although will this leave a jaggy result?)

    (hopefully that makes sense)
  • Vrav
    Options
    Offline / Send Message
    Vrav polycounter lvl 11
    I've always wondered this as well. In a png with true transparency, you should be able to antialias against nothing - thus allowing a clean edge on any background. However, I have no idea. I would personally render an aliased image, magic wand out the background colour and size down the render in photoshop... because I'm a hack.
  • pior
    Options
    Offline / Send Message
    pior grand marshal polycounter
    so yeah thats obviously not gonna work at the moment
    on the RGB image with the pink background, you need to have PURE pink on the edges, not a blend of colors.

    I dont know how to explain it more hehe. If you do sprites work, you dont want gradual transparency, you want a on/off mask,
    So obviously your source render shouldnt have antialiasing on its ege
  • kodde
    Options
    Offline / Send Message
    kodde polycounter lvl 19
    premultiply.jpg

    For best results use scenario 1 with a known background color which you match in your 3d app. Otherwise consider one of the other two.
  • shardy
    Options
    Offline / Send Message
    shardy polycounter lvl 18
    thanks, lots to try!
  • pior
    Options
    Offline / Send Message
    pior grand marshal polycounter
    honestly you guys make is look more complicated that it seems. Just adjust the render settings so that it looks like that third picture right from the start (aka no antialiasing) and thats it. Background = masking.
  • kodde
    Options
    Offline / Send Message
    kodde polycounter lvl 19
    Pior> I should have made a fourth image showing scenario 1 but with matching background color from the 3D app. It would have produced the most pretty looking results, even with a 1-bit alpha. Although you are of course limited to using the resulting sprite on this specific (and similar) colored background or face horrible pixel border artifacts.

    Either way imo it's better to educate people as well as giving them a fix. ;)

    EDIT. Well actually, probably pretty similar to scenario 2. But having 1-bit alpha is more file size efficient, so it's an option to consider.
  • commander_keen
    Options
    Offline / Send Message
    commander_keen polycounter lvl 18
    What are you using the sprites for, what engine? I gave an explanation of whats happening and how to "fix" it here: http://boards.polycount.net/showpost.php?p=1043706&postcount=9
  • pior
    Options
    Offline / Send Message
    pior grand marshal polycounter
    But why talking about premultiplied stuff? If we are talking about sprite authoring like the OP does then its all about 1bit alpha masking.

    It seems like we are confusing alpha blending like in compositing package, with sprite making.
    For 3D sprites I think its simple really : 1 - render the cutout with anti aliasing off ; 2- render the 3D with shadows, fancy materials and so on with anti aliasing on, using a background color consistent throughout the game ; Mask 2 with 1.

    right? Or I am missing something?

    lesprite.jpg
  • kodde
    Options
    Offline / Send Message
    kodde polycounter lvl 19
    Pior> If you set up your renderer to render to 1-bit alpha then I think premultiplied won't have any effect. If you cannot set up your render to actually render to 1-bit alpha and somehow afterwards generate/construct your 1-bit alpha manually then you probably should disable the premulitplied in your color pass. It's hard to say but my guess is it will make a difference. Once again, not familiar with 3DSMAX.

    Judging from your screenshots I think your would benefit from trying it out, it seems like your have the "premultiplied" effect going on there.
  • danr
    Options
    Offline / Send Message
    danr interpolator
    Er ... Unless I'm being a total spanner and am completely missing something, render with "Don't anti-alias against background" checked
  • commander_keen
    Options
    Offline / Send Message
    commander_keen polycounter lvl 18
    well if hes rendering sprites from 3d models instead of painting them by hand he would want AA and transparency I would think.
  • pior
    Options
    Offline / Send Message
    pior grand marshal polycounter
    Commander, really? I cannot tell a single game doing that. All the ones I can think of go for masking. Donkey Kong country, Diablo, even highres stuff like Blazblue or KOF12 (okay thats handpainted but still, very high resolution sprite work) use hard pixel masking.
    Maybe Critter Crunch? even that one Im not sure.
    Honestly it would cause way to much of an headache.
  • commander_keen
    Options
    Offline / Send Message
    commander_keen polycounter lvl 18
    really it would be less of a headache. All you have to do is fix up the rgb pixels in the engine while loading them. doing it without smooth alpha means you have to deal with all the stuff talked about in this thread, and you still end up with a low quality image that can only be used for hard surfaces.
  • pior
    Options
    Offline / Send Message
    pior grand marshal polycounter
    But can you think of any sprite-based game doing so? I agree about how it could look better, but the point is ease of production, touchups and so on.

    Maybe Braid?
  • commander_keen
    Options
    Offline / Send Message
    commander_keen polycounter lvl 18
    yeah Braid, Plants Vs. Zombies and World of Goo (although maybe a good part of it is vector) are some I can think of. Most newer 2d games that are rendered with the gpu would use soft alpha unless they are specifically going for an oldschool look.
Sign In or Register to comment.