The cult theory is spot on, I think. Additionally, I'm getting hints that both MC and the "consultant" have mind control powers, which could be how the cult gets its members. Maybe they both came from different cadet branches of the cult leader's clan and are supposed to be candidates to inherit the cult. As for the reason why that place has so few men, I guess the clan's powers are just more effective on women.