It does refresh but only if you change the file name; usually my in-game item testing ends up with a bunch of files like this: asd asd1 asd12 asd123 asd1234 And so on. Or if you want to keep your folders clean you can just close Dota 2 and open it again and use the same file name over and over.