Y'know, this is a very unreasonably extreme reaction. I mean, those guys were aggressively approaching the girls, trying to force them to do something they clearly didn't want to do, and even physically grabbed onto at least one of them. So Akashi stepped in to defend them. And they treat him like he's scary when he just stopped them from being harassed while he himself has been nothing but helpful and nice to them? Admittedly, sure. He shouldn't punch first and find another way to get the guys to stop. But that's simply the reaction he has when he sees someone in danger. It's just unreasonable.