Hi, So after going through so many blogs and forums related to Unity lighting artifacts (specular wobbly, dark spots, seams, shadow acne etc) I have finally managed to get some acceptable result.
Will be posting the full workflow guide. Used Directional Specular mode with real time resolution of 1 and baked resolution of 30. Had to do some custom lightmap parameters to get these results.
Its done on laptop so on PC quality can be increased a lot though my intention was to keep real time resolution to minimum while getting cool results.
Total 12 assets with 3 uv unwraps each (uv 0 albedo, uv1 lightmap and uv2 realtime lightmap), for PBR materials I used textures from Camoflaj, Viking Village demo and Shader callibration scene.
Unity forums link
http://forum.unity3d.com/threads/arch-viz-guide.380384/#post-2469422 


(This is done purely to get a better results in lighting, specular highlights and shadows)