It's somewhat predictable that the sword lady would refuse. From the beginning, it was shown to us that she's always trying to improve herself.
But I bet this will not last forever, probably they will reach a situation of life and death that requires them to give in.