From what I gather, similar to everyone else here in the comments. Prince Erik seems to be unable to understand those around him and assumes that the raised bar for what is expected of him is carried over and presented to everyone else around him.
As we seen when the Saintess talked about the abusive education she received (which possibly been a lie, or been exaggerated to try and gain the prince's sympathy) he's the only one not to. Only showing confusion thinking that's to be expected. I think, his inability to understand his peers is because he genuinely believed that his upbringing is normal, that and he seems to have a severe lack of being clear with his intentions and way of speaking. Causing the initial "rumored" affair within the 1st & 2nd chapter. Hell, he even admits one of the reasons he likes his fiancée so much is that she's able to understand the intentions behind his words without stating them clearly outright. So his behaviour in speaking to her bleeds to others, which gives everyone the wrong idea and vice versa gives himself the wrong idea about everyone around him.
This is also probably why he chose such poor aides, not simply because he believed that due to their family status and feats. It's more so that he absolutely knows nothing about them as people. Because he doesn't communicate enough to do so, leading him to have the wrong expectations and perspective on those around him. For example, he assumes Sauri, the brother to the more intelligent Bernadetta Mericanto is is equally if not possibly more intelligent, not by being around him getting to know him but by assuming via the short interactions he's had and the background that was presented to him without digging further.
As well despite being prince, like the other boys like Sauri he probably doesn't know that the Saintess will live basically as a prisoner/servant to the church, only existing as a figure head and being forced to give birth to another generation to preserve a bloodline. Which is in turn the most likely reason she tries to get close to others, especially the prince and his aides. Because (in her mind maybe) if someone with enough authority falls for her, they can use said authority to save her from a role she probably doesn't want.