Maybe Japanese readers also kept pointing out how ridiculous the absence of men is, and the author had no choice but to mention it in the story, even if to no effect whatsoever.
I like how it doesn't cross his mind that even with more men, all of the women would still rather bed him.
Like with Peter Grill, women in a fantasy setting want the strongest genes passed on. And there's no stronger than him. No amount of men will ever change that fact. He's the patriarch and always will be.