Another case of isekai as a punching bag.
how that scene should have gone
Knight: I want you to lead us to the bandits
MC: Everybody wants something
Knight: You have to help they are evil don't you want to stop evil?
MC: You mean they might do something like dragging me off to prison when I show up on their doorstep. hit me on the head, drag me around in a bag, and finally demand I do ridiculously dangerous things for them?
Knight: Exactly, they're evil