In a way, I understand the dude's mindset of leaving someone else behind (see other series like Goblin Slayer... if you don't run by yourself, everyone dies)
But on other hand, there may be a chance of actually escaping together, especially if there's no other stronger monsters in the way (since you're heading upwards, and weaker monsters)
They should have at least given her the sword back... (tho I don't think the dudes thought about that as they were only thinking of escaping the dungeon)
So the dude's aren't at ALL faults, but just partly because they "forgot" to give the sword back.