Errors in render output

lpcstr

I just started experimenting with UDK again, and I'm noticing some artifacts in my render. I just did a clean installation of the November UDK with a blank project, and so far all I've done is set up a post processing chain and created an experimental shader. The shader converts the output to look like terminal ASCII art. I'll go ahead and explain the artifacts I'm getting with images.


http://s15.postimage.org/5adluii6h/example1.png

In this first image, you can see along the top and down the left a 1 pixel wide artifact. It looks as though the original image is leaking through here. You can maybe see it better in the zoomed section.


http://s8.postimage.org/cauw6rb7p/example2.png

This one is not visually noticeable, but I picked up on it in Photoshop. I manipulated the image to bring out in red the areas where there is some deviation. Basically the areas in red have one or more of their color component off by a value of 1 or maybe 2. It's sort of a splotchy pattern that seems to form around edges.


http://s7.postimage.org/60ev6oxqv/ascii.png

This is the texture which stores my bitmaped fonts. I'm basically just sampling the Scene Texture, and using the luminance value to index into this texture the appropriate ASCII character and sampling to display on the screen. The font texture only contains 100% black and 100% white pixels, and everything is 1:1, so the output image should also contain only 100% black and 100% white pixels. I have no idea what is causing these "splotches".

I don't think that my shader code is very relevant so I'm not going to bother uploading diagrams and code. Other than enabling my post processing chain under world setting, I have not done anything else to this project.

Ace-Angel

I can't see the first image, but from what I can gather, especially from the second image, it seems like it's a case of bleeding form the luminance map, or lack of aliasing (which I'm guessing is a UDK limitation)?

When you're sampling your luminance, how are you clamping the values? As you raising them to a high power value to create the different values as masks?

lpcstr

Ace-Angel wrote: »

I can't see the first image, but from what I can gather, especially from the second image, it seems like it's a case of bleeding form the luminance map, or lack of aliasing (which I'm guessing is a UDK limitation)?

When you're sampling your luminance, how are you clamping the values? As you raising them to a high power value to create the different values as masks?

Maybe you can see it here: http://oi46.tinypic.com/r0rssx.jpg

I really don't know what you mean by "luminance map."

My post processes has two materials. The first uses ScreenPos and ScreenSize nodes feeding into a Custom node with some quantization to produce a UV which gets fed into a Scene Texture Node, the result of which is to pixilate the screen, so that each pixel appears to be 8x8.

In the second material I have a Scene Texture Sample node that samples the pixilated scene, i then take that to a Custom node, where I perform a DOT of the pixel color and [0.299, 0.587, 0.114]. I then take the luminance value, and I quantize it so that it corresponds to just 16 levels of grey. Then using a ScreenPos and ScreenSize node as inputs and my quantized pixel luminance I calculate the correct UV values to index my font image. The output of the font's Texture Sample is then output as the final pixel value for the screen.

I'm not sure what you mean by aliasing either. Like I said, there is a 1:1 ratio of font texels to screen pixels.

lpcstr

Just to prove it's not anything in my material that's accounting for this.

Here are the results of setting up a post process chain with nothing but a single material node.

In that material I have nothing but a single Custom node connected to the emissive channel.

The HLSL is just "return float3(0, 0, 0);"

And here are my results.


http://s20.postimage.org/h6nekk2i5/example3.png

This is 1280x720 contents of the window. Should be just a solid black screen. But it's not!

You should be able to plainly see that there is a 1 pixel wide border along the top and left. You should be able to tell that what is happening here is that the post process effect is not being applied to this part of the screen at all.

Next, you should be able to assume that every pixel in this image will be pure black (rgb [0, 0, 0]). Nope!


http://s20.postimage.org/92faftg31/example4.png

This image has had it's contrast adjusted in Photoshop to emphasize the random (imo smudgy looking) artifacts around the screen. In reality the largest error is a pixel value of rgb [4, 4, 4], which isn't perceptible to the eye, but it's still a long way off from [0, 0, 0]!

Seriously UDK, what gives?

Angry Beaver

there's some bathroom tile poking through the top left, are you sure your thing(overlay, stencil shader, what is it?) is aligned properly, maybe you want to oversize it a little and offset it?

lpcstr

Angry Beaver wrote: »

there's some bathroom tile poking through the top left, are yuo sure your thing is aligned properly?

Is my what aligned properly?

Ace-Angel

It honestly is an UDK issue, it has clamping issues when referencing a scene at the edges of your post-process.

Here is an example for cubemaps issues, it should be the same 'idea' behind your PP: http://udn.epicgames.com/Three/DevelopmentKitGemsCreatingDistortedReflection.html#Clamping%20UV%20coordinates

Did you try and see if you run the scene in a 'game' instance (not the editor) alleviates the issue?