Yeah, I've found you lose a lot of value when doing "hue/saturation" (which has the same results as 'desaturate') The best solution is to do what I did, and make an action that creates a new layer, fills it with black, sets it's transfer mode to color, and merges it down.