You might need some additional support loops there. Chamfer modifier does help alot in making the high poly but is not a one-click solution alot of the time.
That's an expected result based on your topology. You can probably get something working with just the chamfer modifier. Turn off min angle and allow loops to traverse across some of the flat surfaces if you have to.