You know what actually annoys me a lot about this? It's the fact that the author/artist hides his unattractive face so much by purposely just not drawing it. Literally everyone thinks that his face looks scary. That includes all of the dorm residents, and it's briefly shown here in this chapter. It's also a major plot point about the character and his personality.
But the author/artist specifically does not show that to the readers most of the time to try to get the reader to not think about it.