No they probably are not the ones in the position of power. Men are still stronger than women in this world it looks like.
Physical strength isn't the defining issue that would make Women the dominant force. It's the fact that if you let the 20% of men in your society die, you're doomed. For example, construction kills at least a thousand a year. If your population has 50K people, with a 1 in 5 ratio, 10,000 of them will be men. That means in 1 year, you've lost half of your societies already dwindling ability to breed. That means instead of a simple 80% never being able to reproduce, you're now stuck with 90% and growing
With how heavily the human population relies on men, they're not going to be anywhere near anything that could even be considered slightly dangerous. Women would have to step up and they would have to take charge and keep their men safe.
Now, I'm not disagreeing with you. Men being stronger and dominant is probably true for this manga. But I reiterate that my main complaint is that the author isn't letting the concept speak for itself. He's using it as a thin veil to tell a generic harem story, while ignoring the logistical and logical issues that stem from a population of 1 in 5. So yes, I fully expect things to be as "normal" as they are here on regular earth.
Edit: Calculation correction
