Fixed part of it, the artifacts of the lowpolygon model are fixed by turning on AND off "Fix Mirrored Tangents" in marmoset toolbag. The bad bake in xnormal (see the first normalmap) logically did not change.
See gifs below, My lowpoly faces are somehow baked onto the normal map when using Xnormal, and are showing up in Marmoset Toolbag 3. I was wondering if anybody knew how to fix this. When I use the "use average normals" setting when exporting in sbm or in xnormal itself the result is fine. (Which is not a problem in this…