That was an interesting bit explaining why he's so attracted to Huahua: She's the one who behaves the most like a girl from his original world. Of course she's cute as well, but nevertheless, it seems like she's the one he ought to have the most genuine feelings for. Ye Chenxi he only likes because of her body and family connections. He's attracted to Tang Ziyan solely because her counterpart was his wife in the previous world, which, basically, means he loves only a ghost, nothing more, in her case. Of course, just like in Ye Chenxi's case, he's also profiting from Tang Ziyan's family background.
No matter how you look at it, Huahua is the most precious one. Sure, she helps him, which is natural, but it's nothing special he couldn't replace, and her limits are clear. So, he's not going back to her only to exploit her background or anything. For Ye Chenxi the MC used to be a boytoy she wanted to rape, then the savior of her mother, but that's more or less it. I haven't got the bloodiest of idea why Tang Ziyan bothers to stick to the MC in the first place. Perhaps because he's an unusual man, who knows.