@criver
While I would say that the garrison would usually go out to attack a smaller force (breaking the siege), the demon army was expected to be much larger.
However, just like how the advent of gunpowder made large stone walls obsolete, the demons have these golems and magic that could probably break walls anyway. Some of the demons could probably climb those walls very easily without a ladder, and certain formations and tactics devised against those demons probably won't work in tight spaces either. They also have flying demons, and the best way to counter them while reducing collateral damage (to the buildings behind the wall) is to intercept them as early as possible.
We don't how much thought the writer and the artist put into this, but I think this is something that can still hold suspension of disbelief. Speaking of the writer, it's the same guy who did Konosuba, which also depicted fighting demons outside the city walls (in the anime); it may have very well been thought out.
Edit: Oh so a golem really did easily break the wall