If the guy back in middle school was seen as popular at the outset (he's physically attractive and charming, so I think it's a safe bet that he was), then he would likely have had the social credit to be able to spread rumors and torpedo Hajime's reputation, and everyone would gravitate toward him & his word because that's how popularity works.
Especially if it was an established status for him by the time whatever transpired for Hajime occurred. If he was popular in sports (basketball being a running sub-theme, I could see that being the case here), and reasonably good at it, all of that combined with his charm & looks means he carried emphasis among the student body, and hangers-on who wanted to be liked by "the popular kid and his circle" would bend in favor of going along with the rumors and perpetuate the ostracizing of Hajime.
And whatever the actual context - Hajime being responsible for his younger sister's death would be a big deal and very dramatic. Rumors being what they are, all you need are a couple slightly-varied stories going around, and suddenly they become self-corroborating and Hajime can't push back against all of it on his own.
Such is the way of peer pressure in the social gauntlet that is school. This is honestly entirely believable, and leads me to think that Hajime was thoroughly pushed all the way outside the social system of his old school, which goes a long way to explain his lack of trust, insular personality, "masking" constantly at school, and his declining to push back the whole "#2" thing.