I'm pretty sure that's not possible. The only way to do it correctly is through a post-shader. It's because you need to compare a pixel with bordering pixels, something that can only be done when sampling from UV's, which is obviously not the way to go. In Zbrush they do everything on CPU plus they can work with vertices…
and now how about making it usefull with atleast a diffusemap and such stuff, i still didnt understand the math really (from what i know is retransformating the normals from tangent space to camera space so that the normal vector points to the correspondending pixel in the spheremap) i tried to acomplish this in shaderfx…