Evacuate the whole town and build a new town at a safe distance. Future generations will be grateful. It would be an interesting dungeon as it would contain the old town. It's not like the fresh dungeon would immediately be so dangerous folks would need to leave without all of their possessions.