He's finally come to realisation that he only likes her in a sexual way, and therefore would not be a good match for her. Plus, while she's not careful in being overly nice to him, he can upset her in a big way. It'll become clearer what changes this makes over time.