What? They show the two male maids talking about it. They are the stand in for the public opinion on male beauty. Faust is a war hero, his physical appearance matters very little. He's still attractive, just not conventionally attractive by Anhalt standards. The merchant knows that his availability is essentially zero.
If we had joined him before then, sure. But you have two of his compatriots, the 1st Princess and the Duke Astarte, who are attracted to him. Even in chapter 1, we get the knowledge in the Manga that the fact he resembles the former king is part of why the Queen is attracted to him. Him not being conventionally attractive to the Queen, the Princesses, and the Duke is not a problem. The Queen, Princess Anastasia, and Princess Valerie won't search for a wife for Faust for obvious reasons. And him being essentially marked by both the current and future monarch, let alone the Duke who is 3rd in line from the throne, ensures that any robed noble who might find him attractive is warded away. No one in government is going to petition the Queen for a family member to marry Faust.
Faust is a border lord of a poor region that is resource rich. Valerie's royal guard are all daughters who no longer can inherit their family's title for one reason or another. They are all thirsty virgins. Faust is a pretty great catch for them. I'm also not sure the members of the guard actually care that Faust is being spoken for by the Queen/Princess/Duke. And Faust doesn't exactly realize that at the moment, either.