The MCs goal is to for her family to live happily not to get rid of the Mistress.
That's all she cares about.
I'm thinking she won't do anything until the Mistress proves to be a danger. Cause the king could always retaliate and harm her family.
Also there's contraceptives, I'm assuming, in this world. Back then (in this world) there was some form of contraceptives so why not that world. So it's not impossible to prevent a pregnancy though obviously the Mistress doesn't want that.
Though considering she came from a nobility I don't understand why the king couldn't bend the rules to make her his queen. It wasn't like she was a commoner.