Then you leave the country bro.
Also, "living in peace" as a battle/sex slave is quite the perspective.
You're forgetting a teeeensy detail... All the other nations around this one do exactly the same, or worse..
That "Garbage" is created by tossing
summoned who are no longer needed ( or presumably become too dangerous to handle/control ) into a "demon realm" to play a nice little Death Game...
Same as killing the Chancellor.. He could, easily. There's just the tiny, insignificant detail that the Chancellor is the key to the slavery magic controlling the ladies, including his wife..
And there's no telling what will happen if the Chancellor suddenly expires. It
might be that the magic gets cancelled, but MC is familiar with the standard slavery contract tropes and realises that there's very good odds that there's a kill-switch involved.
So the Chancellor lives, for now.
Same with the grouching that MC "does not care for his classmates".
He does, and is actually depicted aiding his classmates in meeting their quota and keeping them out of harm's way as far as he's able and present up to the point where they
should be able to take care of their own problems.
But there's only so much one man can do, and his priority is his waifu. Period.
So yes.. given the alternatives, living a relatively comfortable life as a battle/sex slave is the only viable option. For now.
And he's using that option to the best of his ability, using that famous japanese deadpan face that does very well to hide his feelings and plans.
The premise of the story is your usual Trash™, but there's logic behind the madness.