I don't think it's ever clarified, but Ferdinand is just conforming perfectly to the beauty canon of the world, plus he had to be charming to survive due to his situation while growing up. There is a short chapter that showed how he had to hide his true self to Veronica since he was a child.
There is also the fact that Elvira has a lot of imagination as the later chapters will show
so Ferdinand's distance from the rest of noble society is perfect for her imagination to run wild. Iirc there is also a story from his time at the academy, but I don't remember the details/don't know if it's ever been revealed in details.(she is THE most successful and prolific writer of romance novels when she begins to be involved in the printing industry)
I see, thank you. I'm a bit disappointed it's the same "because of his look" just like the other women.Ferdinand is the "JPop/KPop" idol of that country, and it doesn't hurt that he's her husband's cousin, so she has that forbidden relationship fetish working full time in her imagination.
Myne's adoptive mother is not this bad, but Elvira is really down bad.