The places are ao cheap, because you need to work your ass out on the fields and it isn't guaranteed to pay off, sudden weather change for instance, then there's no 100% certainty that the products will be sold.
The town depopulate because they don't have enough valuable or demanding products to sustain in the long run.
Anyway, we all know where this is going, meh...