20-40 materials is (to use a technical term) fuck all - You have nothing to worry about. Assuming you're in Ue4 look at the stat console commands to see draw calls/frame times etc. Don't get too hung up on it though, if you've done a village with 20-40 materials you're probably doing it right
Sharing materials across multiple buildings is generally a good approach. You can also model elements in a modular way so that you're reusing meshes in addition to materials. While each building may have 10 materials, objects sharing a material will be batched into a single draw call. It can get more complicated than that,…