I have a staircase asset that I made for an environment in UDK. For some reason the normal map for this object looks very pixelated and ugly when brought into the engine and applied to the object. I imported it as normal map compression and chose world normal map in the import settings. I sculpted the detail in Zbrush and took that high poly and baked the details onto the low poly using xNormal. I made sure to flip the green channel, so far all my setting have been compared to alot of others setting in different tutorials but the normal maps still dont show up looking the way they should in UDK. I'm using a 2048 x 2048 map as well.
You can fix the pixelation by choosing the "NormalMapUncompressed" compression method. This uses more memory, but looks better, so you need to decide if it is worth it. Flipping the green channel depends on what you baked the normal map with, it isn't always necessary. UDK requires X+ Y- Z+, similar to 3ds Max.
Edit: Just read that you used xNormal to bake. If you baked with the default settings, flipping the green channel was correct. Did you use a cage to bake in xNormal?
From your normal map, I assume you're using hard edges along the stair steps. You should separate your UVs wherever you have hard edges, otherwise you will have issues. You'll also want to use a cage to bake, otherwise during baking some of the rays will miss parts of your high poly, making your edge seams really stick out. Look at this thread here.
Replies
Edit: Just read that you used xNormal to bake. If you baked with the default settings, flipping the green channel was correct. Did you use a cage to bake in xNormal?