Oh, it's even better than that. Remember what the knights said a couple of chapters ago?
He's letting them go so they can come back and attack the people he just saved from slavery, along with their families and homeland.
Couldn't post last chapter before it got locked:
Anyone one else think it's hilarious that Zect spends all that time reflecting on how he has to change and become a better person, and then immediately thinks "except for doing my own chores. I'll buy a slave for that."?
I feel like the MC could have just said something like "before I go, can you just relay this one message to him?" and then told them something that would have let Gurba know exactly who was outside the door.