borders on rendered transparancy

shardy

Im having some issues with borders on rendered transparency I am using 3dsmax.

I am rendering out some 3d into sprites, they are rendered with a transparent bg however its apparent that the frames all have slight bordering around the actual 3d model of the 3dsmax environment background color (pink in this case).

Is there something I can do to avoid this happening?

Thanks

kodde

Pre-multiplied alpha, try turning that off in render settings. It ought to remove what you are describing, but you might get slightly more "jagged" edges.

Will your sprites be used against a solid background color always or varying colors? If it's a constant solid color, just match that in 3DSMAXs background color in the renders and you don't have to bother with this issue at all.

shardy

Thanks for the tip, background wont be a constant solid so cant do that

Will give it a crack !

kodde

Actually, it doesn't have to be "constant solid", if it's generally one color all over (dark for example) you can get away with this. You will actually get more smooth transparency if you can keep this effect and use it to your advantage.

Either way, try finding this feature, render with and without to see the effect.

Good luck.

shardy

forgive my ignorance, but where can i find it to turn it off

pior

dont bother with saving the transparency as an alpha channel

In case you are applying a texture with transparency :
- Make your diffuse a flat bitmap (RGB), and make the surrounding color around the sprite fairly dark
- Create a separate blanc and white image for transparency, also without alpha channel. Load that one in the transparency slot.

In case you are rendering something and want to make it a sprite :
- render against a dark background,
- render another black and white pass for he masking, and keep that pass separate for compositing

Easy, and impossible to fuck up!

shardy

Wouldn't that mean id have to manually apply the b/w pass as the alpha in photoshop for each sprite?

kodde

pior> In your second scenario wouldn't you want to turn off the premultiplied alpha effect either way as to not get the background mixed with the border pixels? At least in your color pass.

EDIT. Unless you could possibly turn off Alpha all together and having the rendered therefor not considering premultiplying at all I guess. Not familiar with Max.

pior

wait- maybe we are talking about different things.
Could you post a picture of what you are trying to achieve. For instance, you could be trying to do sprites with 'solid' masking, meaning that a pixel is fully transparent or is not, but you end up fighting against the antialiasing of your renders?

Please post pictres of :
-your current result, broken between RGB and A passes
-an example of the type of sprite transparency you are after.

Kodde : The reason why I always separate RGB from A in two images in every transparency work I have to do (either compositing, or applying transparency to a texture) is to avoid exactly this kind of questions :P I don't want to battle against file formats or terminology, so I just screw that and work the easy way.

Not sure what you mean with background mixed with border pixels. When rendering sprite from 3D you dont want any antialiasing on the edges anyways so why would you get any color mixing? You lost me here.

shardy

hrm i'm not surprised, i'm not super clear with my explanation




I have cropped and zoomed in a bit into a portion of the image, in 3dsmax i have my environment bg set to pink (255, 0, 255) and have rendered out the character as PNG with alpha, as you can see the character has a border the same color as the background.

What I am after is nothing surrounding the 3d model, just the model itself rendered out with what I assume would be a hard transparency? (although will this leave a jaggy result?)

(hopefully that makes sense)

Vrav

I've always wondered this as well. In a png with true transparency, you should be able to antialias against nothing - thus allowing a clean edge on any background. However, I have no idea. I would personally render an aliased image, magic wand out the background colour and size down the render in photoshop... because I'm a hack.

pior

so yeah thats obviously not gonna work at the moment
on the RGB image with the pink background, you need to have PURE pink on the edges, not a blend of colors.

I dont know how to explain it more hehe. If you do sprites work, you dont want gradual transparency, you want a on/off mask,
So obviously your source render shouldnt have antialiasing on its ege

kodde

For best results use scenario 1 with a known background color which you match in your 3d app. Otherwise consider one of the other two.

shardy

thanks, lots to try!

pior

honestly you guys make is look more complicated that it seems. Just adjust the render settings so that it looks like that third picture right from the start (aka no antialiasing) and thats it. Background = masking.

kodde

Pior> I should have made a fourth image showing scenario 1 but with matching background color from the 3D app. It would have produced the most pretty looking results, even with a 1-bit alpha. Although you are of course limited to using the resulting sprite on this specific (and similar) colored background or face horrible pixel border artifacts.

Either way imo it's better to educate people as well as giving them a fix.

EDIT. Well actually, probably pretty similar to scenario 2. But having 1-bit alpha is more file size efficient, so it's an option to consider.

commander_keen

What are you using the sprites for, what engine? I gave an explanation of whats happening and how to "fix" it here: http://boards.polycount.net/showpost.php?p=1043706&postcount=9

pior

But why talking about premultiplied stuff? If we are talking about sprite authoring like the OP does then its all about 1bit alpha masking.

It seems like we are confusing alpha blending like in compositing package, with sprite making.
For 3D sprites I think its simple really : 1 - render the cutout with anti aliasing off ; 2- render the 3D with shadows, fancy materials and so on with anti aliasing on, using a background color consistent throughout the game ; Mask 2 with 1.

right? Or I am missing something?

kodde

Pior> If you set up your renderer to render to 1-bit alpha then I think premultiplied won't have any effect. If you cannot set up your render to actually render to 1-bit alpha and somehow afterwards generate/construct your 1-bit alpha manually then you probably should disable the premulitplied in your color pass. It's hard to say but my guess is it will make a difference. Once again, not familiar with 3DSMAX.

Judging from your screenshots I think your would benefit from trying it out, it seems like your have the "premultiplied" effect going on there.

danr

Er ... Unless I'm being a total spanner and am completely missing something, render with "Don't anti-alias against background" checked

commander_keen

well if hes rendering sprites from 3d models instead of painting them by hand he would want AA and transparency I would think.

pior

Commander, really? I cannot tell a single game doing that. All the ones I can think of go for masking. Donkey Kong country, Diablo, even highres stuff like Blazblue or KOF12 (okay thats handpainted but still, very high resolution sprite work) use hard pixel masking.
Maybe Critter Crunch? even that one Im not sure.
Honestly it would cause way to much of an headache.

commander_keen

really it would be less of a headache. All you have to do is fix up the rgb pixels in the engine while loading them. doing it without smooth alpha means you have to deal with all the stuff talked about in this thread, and you still end up with a low quality image that can only be used for hard surfaces.

pior

But can you think of any sprite-based game doing so? I agree about how it could look better, but the point is ease of production, touchups and so on.

Maybe Braid?

commander_keen

yeah Braid, Plants Vs. Zombies and World of Goo (although maybe a good part of it is vector) are some I can think of. Most newer 2d games that are rendered with the gpu would use soft alpha unless they are specifically going for an oldschool look.