using smooth normals works when everything is perfect but at no other time. in practice, if the model looks good in an untextured, shaded view then you're in a good place to start baking. it's not a guarantee that hardening all your shell edges will look the best but it's a good starting point
I just baked a segment with edges given the 'harden edge' option in maya and the normal map came out much better, im slightly confused however but after watching some videos if I understand correctly I should smooth all the edges except the hard ones? I was under the impression you wanted to smooth the entire mesh.
Smoothing entire mesh works ok for organic subjects or characters. Anything boxy shaped or just having close to 90 degree angle in between polygons would need either split/hardened edge or a geometry bevel/ chamfer with vertex normals slightly rotated to be perpendicular to adjacent bigger faces . It's called face…
Your problem is on modelling side. I see same huge shading gradients on your mesh too. And while normal map can theoretically counter-measure them in a perfectly "synced" workflow you would never get it looking perfectly right in actual game anyway . 8 bit normal map with compression just doesn't have enough precision for…