She was indeed ugly...atleast in their world apparently. If the average/plain girl in an anime/manga word had a big eyes and a slim nose, aka cute but not too sparkly and here we have an mc who looks a bit more realistic, then yes, its not far fetched to call her ugly. And no, if you're going to argue that no one is ugly, everyone is beautiful or such cheesy words then dont, when they said ugly, they litteraly mean what the word is.