Is anyone here reading the story? He literally can't have a relationship, this is the condition his parents gave to him, if you want to hate someone, hate his parents.
He clearly rejects her, more than once, so nobody can blame him for this part.
Yeah, he sure messed up letting her wait, but he...