I'm going for rough where wear would occur. I want to give the impression of a cared for but old rifle, hence the deep grain, rings and scratches. I think you're right on the brightness of the diffuse.
I had a deeper thought about the dirt layer and realised that by multiplying an AO pass with some existing masks & a noise based procedural diffuse I'm able to produce a pretty good 'nook and cranny' dirt layer.
I would recommend getting the Marmoset Toolbag. I used to use the xoliul shader, but it gave me some nasty looking colors. (pink and purple when I know that there is none in my diffuse texture) Not trying to make the xoliul shader look bad, just might want to look into the Marmoset Toolbag. Your normals are too thick…