handplane supports 16-bit input and output so I would try re-baking at a higher bit depth and see if that fixes the problem. To set this up for xnormal go into the image exporter settings, select the tiff format, and then hit the configure button.
Its a 1024 map and it did had 2 in padding and i tried 16 . I did also bake a 2048 map but its still there. Handplane seems to remove the padding, probably doing it with the unwrap and it seems like the normal map is inside of the unwrap like a pixel or so on some spots and outside in some others.