Author is being absurd here. Feels like he went "oops went too fast on the 'Humiliation of the Hero Party' trope! I'll go redo it."
Ceres has already bypassed them in everyway. He's not just the leader of the new hero party, he's a one man hero party. (Despite how skilled his milf wives are, he doesn't really need them in fights.) And he just got done rescuing the former hero party when they completely failed, got captured and were about to be sacrificed. Plus he's screwing their moms.
Even Zect shouldn't be delusional enough to think that Ceres should "come back and join his party".