@Amerak, this is more of a slice of life manga.
Nothing is bound to really happen unless there is a sudden genre shift.
Note how the very few battles were very short and totally one-sided.
This is a story about an isekai'ed protagonist slowly build his domain (from a farm to a village... and now moving towards a small kingdom) while building relationships with his neighbors rather than conflicting with them.
If that is not your kind of story, that's definitely going to disappoint you. I do enjoy battle manga, even some with op protagonist (though that's being overdone by now), but some variation is not bad.