yes dont have black in your diffuse, almost never have black in your diffuse, as when this lights you will get large areas of flat grey, think about whatthe material is when lit and go for a darker more saturated version of that instead
Yeah, what sepeiro said. You don't usually want any pure black (or white for that matter) in your diffuse because it gets multiplied by the other maps. Ian Joyner mentions that briefly in his Gnomon texturing tutorial, its character stuff, but worth checking out imo.
Thanks for the feedback guys. I forgot to mention that I took my actually stone pillar into zbrush and did some sculpting. After I textured, I took my diffuse into crazybump to pull out some extra details, then did an overlay in photoshop. Here is the updated normal. It has colors blown out, why is that? What is causing…