Seing how he was being talked down like he couldn't make it by the teacher, it's obvious he wouldn't want to come back if an issue arised. All of this is totally deserved.
Looking at mc's life story before that, why would he be the one who has to be the better person first ?
Anyway the same could be said for Ruo and his family, it's just that it feels a bit unfair from mc's point of view since they were so arrogant towards him. They were right, they had their reason to be harsh, but it still hurt anyway.