Yeah, it's weird. He blames Yona for his wife's death, which is foolish. It's certainly the reason he neglected Yona. However, even if he refused to see Yona, it seems odd for him to fire the physician if Yona still needed treatment considering that he seems to value his promise to his dead wife. It's possible someone, such as his current wife, interfered or fed him incorrect information about Yona to get rid of him quicker.
I wonder what Prince Carlo is up to, but it seems to be discussion that shouldn't be eavesdropped upon.