That bandits are stupid. Even thought they can (probably) overwhelm him (which is probably don't) and kill him (also, probably never), they will still stuck in gate and high chance gonna get caught (and probably killed) in process.
I had a feeling this dude was a jealous coward, most likely a traitor! But still I did hope for an Usapp type of comedic relief coward, with redeeming qualities...oh well, let's see why tf the captain still has faith in him