Well, I personally think that's the realistic part of this manga. This manga is consistent with its characters. No one changes from a weakling to a bad ass just because he is getting punched once. Tendo's growth are slow but that's how people in real grow. Even after he went through all those, he is still a weakling. The difference between him before and him when gets revenge is his anger and hatred control him. That really fits his personality that has been built up since the first chapter. This manga is about bullying and tries to tackle many subjects related to bullying, which is one of major problem that's being a concern in Japan. And I personally think this is the most realistic and complete manga that discusses the subjects.
That's just my two cents
Edit: btw, the bullies didn't bully his crush. Tendo is afraid Hirahara gets involved but Hirahara never gets involved because she is afraid Mataga will commit suicide if she does. If there's someone bullies Hirahara, it's Mataga. Tendo actually is glad that Hirahara didn't get involved. But he doesn't know that Hirahara is also struggling with her guilt for Tendo and Mataga makes her unable to do anything