Wow. I'm so surprised [sarcasm] to yet see another protagonist, who's very apologetic, insecure and shows very submissive behavior towards weaker, irrational idiots, despite there wasn't need or nobody asked for it.
He himself is just feeding their retarded beliefs of him, by basically "rolling over", like a dog, when instead he should open his mouth, show his "fangs", and clear that misunderstanding.
"But I'm a monster, I deserve all the hate of the world", is basically this idiot's way of thinking.
Very unlikeable character trait imo. One of the biggest turn-offs as a reader, is to see the protagonist show this type of submissive-/beta like of behavior, though there's no need for it. (the maids are more sane/rational than the mc, since even they thought he was being a retard)
All that's missing now is to see a picture of the author, doing a dogeza pose, and apologizing for this protagonist. That would make a full circle.