Allow me to say something first.
I am a fan of harem troupe, but I am not avoiding non-harem stories... I read this manga at the time it doesn't have 'harem' tag yet.
Now, it will? I don't know... But surely there would be people who will drop it, I am talking about those who purely dissing 'harem' ofc.
If anything, I just hope the author will not regretting any of the choices they would made. Regardless to which way this manga will head towards to.