Here is the thing.... this was a harem because multiple of the girls were attracted to him....
That's all it takes. Even if the MC only likes one girl.
Now, Tonikaku Kawaii doesn't count because the number of girls that liked him was few, but they easily moved on and it over clearly has them as a couple. And the harem-ish element barely lasted for a few chapters among HUNDREDS...
In this, multiple girls want to be with the MC.... and he's not actually with the girls.
Now, if it ended and the girls moved on and this had HUNDREDS of chapters, that'd be one thing... but when the harem element was introduced, it only had ELEVEN chapters... and it had a few chapters of other girls wanting to get into his pants... and that part is still valid till chapter 25....
Harem tag doesn't mean having an actual harem... it just means you have multiple girls want the MC for a SIGNIFICANT PORTION of the series...
Even if those girls have no chance and the harem element is basically pointless and adds nothing to the series....