To be fair, I'd say they were completely complacent as a result of having been known to produce many elite-level combatants for years now. It's likely they haven't been attacked in a good, long time.
Complacency and it's best bud hubris do claim many a victim, but what I see here is an author either being lazy and looking for cheap drama, and or fundamentally misunderstanding how a fantasy society like this would function and manage its defenses.
If the village management went to the trouble of putting up a wall like that and posting a couple of normal guards, then there is going to be some selfimportant elite near by making sure they, and the other guards like them, are doing their job.
Much like in the same way the lizardmen would have a much harder time breaking into the elites house to take hostages. Not only because the individuals in those homes should be inherently stronger, but also because the elites would clear any threats as they moved twords the main gate.