the first arc seeming like civilization was destroyed can easily be explained by "it was"
Asley was holed up in his cave for centuries, and as we can see in the beginning, the civilization near him, that he was probably from, was destroyed and nothing was rebuilt on its ruins. this could be due to changing climate or rumors of the place being cursed or anything. where he's at now is a new civilization that started after his died off