make sure you've set the modifier's luminance mid point to 0.5 ... if it's 0.0 then it will only push out, if it's 1.0 then it will only push in. 0.5 will use mid grey as mid height.
right ive got a nice height map out of Xnormal, when i apply it as a modifier in max i only get the positive normal push, not the negative, am i being stupid, how do you get it to push both ways? I need it to be a modifier so i can optimise the underlying mesh correctly