try duplicating and merging your AO and diffuse layers, for what i can see the magic tool is selecting everything because of the source texture. the AO will add constraints to the magic tool. Also try desaturate the duplicated layer, and play around with the magic tools values.
what ZacD said about the direction of the wood grain. A technique i use when doing worn paint on different surfaces is to use the magic tool in PS with a low tolerance i used between 4 and 6 in the paint over. select on the wood grain layer, and on a new layer paint in a solid colour. use blending modes, saturation or what…