the reason is that you the curve where you have your hole in has less segments than you need for the actual hole. You could probably work with that ammount of sides on the big cylinder. But the hole itself would need to be simplified. Infact you could probably delete that hole the way it is now and just extrude that stuff…
fine then, you know the solution , you are just beeing stubborn trying to make a weird point ? dont add chanfers/control edges until you have a clean base shape .