I think it was either one of or a combination of several things. Didn't want to get his hopes up, heartbroken by her memory loss, maybe stung by her cold treatment of him, or didn't want to potentially start a relationship again when he was the only one who remembered their prior past.