Ooh, now I get it.
He always liked her, even if it was just friendly feelings in the past, before the new soul took over the body. But because of his past he was reluctant to ever show his feelings as he feared it would bring harm and worry onto his wife.