Not really clear who's the harem member, but basically top adventurers, top tier girl, including one US president daughter if i remember correctly.
Regarding his childhood friend, yeah right now she undermine him, calling him jobless, etc, but at some point he was contacted by the US govt about his guide, of course japanese govt soon follow to protect their national interest, and other country soon follow, receiving so many attention, it's impossible to hide his identity, at this point he already billionaire receive so much money, fame and vip and protection status from so many country. Reality crashing down of the childhood friend, realize that she basically throw the jackpot ticket away, now he's at unreachable status she resort to all sorts of sinister plot to get it back to him, including falsely accused MC that baby on her stomach is his, but failed miserably, because everyone know, long way before the childhood friend was pregnant (she's playing with lot of hansome men) MC already cut ties, and no longer meet or talk to her, she also resorted to stalking to the point become wanted criminal and dying in the dungeon while trying to escape police, she absolutely life a miserable life the moment MC cut ties with her.