In Universe, Takae and Ushigome may have bad starts with him (putting it mildly) but that wasn't the point I was making there. Just that Ushigome wasn't the first to confess her feelings to him.
It's been over a year since that happened and while trauma doesn't just go away, he's supposed to be their friend. If his first set of thoughts are doubting Ushigome when she poured her heart out to him he's a terrible friend. On the Prez, yeah she deserved a clean rejection because that's what all girls he doesn't like deserve. He writes off everyone's feelings as "not serious". He doesn't deserve to be with any of them because that's a massive lack of respect.