PBR textures should have no lighting information (With the exception of Ambient Occlusion). This is unlike the old system where Ambient Occlusion would be put into the diffuse map.
Seems cool right now :) . Could you tell us which PBR Worflow your are using ? It seems to me that you are using Specular/Glossiness Stuff. If so your specular have too much color and levels variations, every different pixel intensity is a different material on that map. Your Gloss need to not be just your albedo in black…