The place they live at is supposed to be being demolished within 2 years at the start of the story.
I'm solidly on board with the Husband having an actual family and falsely being married to FMC, but it could also go the route of them actually having an affair figuring out where they go and what to do and then ultimately deciding it was just an affair (lol "just") and leaving when the place is demolished.
What would be worse is if the Husband wasn't having an affair and was working hard to buy a house to have a family in.
Still I think it's the Husband is cheating though.