Large scale AO really should not be baked into specular (or the metallic materials of a albedo map when using the metalness workflow), or diffuse, this just does not make much sense physically. As mentioned above, when you do that, your AO is occluding direct light sources, which is very much the incorrect thing to do.…