I think it's the other way around. She got summoned a long time ago by the evil gods, and became powerful threat. Then the good gods sought out a soul which would be most likely to win against her.
I'm pretty sure that the only thing that's saving the hero party is demon lord Aisu's obsession with destroying the party using seduction. If the MC abandoned his principles here and used violence she would conclude that he's "just like the other men" and easily murder the whole party with...