I know this is pure speculation, but I'm at least highlighting it as such.
I think his inferiority complex is what prevents him from being in a relationship with her. I don't think it has ever been explicitly explained, but that's my fan theory as of now.