Are we reading the same manga (well, in my case reading the raw and translating it)? Chapter one he was being whispered about and ridiculed openly, and again just a few panels before the one you are talking about. He's a "loner" who doesn't want or like attention, and probably doesn't want her...