I take issue with the tendency in Manga to treat "development" and city growth as an unambiguous good. It feels very... colonial. Nobody even bothered to ask the locals if they were okay with a group of strangers coming in and completely changing their community and whole way of life.
in chapter 22, the number of locals is just under 50, dropped from around 100, so I'm sure they actually are glad more people are on the island (chapter 25 also show that the fishermen like the idea by helping out with the building process)
It's also important to note that the noble who own the island doesn't care about it (to the point that it's really the guild that's running it, considering they bought said nobles house on the island), so technically it's a case of the people who basically own the island revitalizing it (The dungeon is the main thing about it). Plus yea, as others mention, they did ask the locals beforehand (granted, they asked for help rather than if they wanted it, but they didn't refuse, so indirect acceptance of the idea).
Plus, it's not like the locals changed their way of life much, p sure they're all still fisherman or guildmembers (those are the only listed people on the island at the start and doubtful that there's any other occupation), and now they actually have a steady stream of supplies because the ships come by more than just once a month (chapter 22 again mentioning that the only people going to the island are merchants for supplies)
EDIT: oh yea, forgot to mention - The guild was going broke, which meant they could no longer help support the locals (Gene is a guild member, but also the link to the locals aka fishermen), so not like they'd really be able to stay without much support (they need money to pay merchants for supplies, and presumably they get money from selling fish to them, but also to the guild). The island was by definition dying (locals leaving the island and funds disappearing), so this is a case where it makes sense for growth to be the goal.