So, I've seen this problem before and thought I knew the solution, but I seem to be wrong!
I thought it was because my UVs were inverted. So I if I mirrored the appropriate shells, this inversion would go away. But now this doesn't seem to be the issue.
The seems are quite visible and I'm not sure why. Anyone familiar with this problem?
Thanks.