Yup, that is what I mean by assumption of this culture.
Plus mangadex definition is taken from Anime-Planet with no reliable references.
According to wikipedia, the word harem is taken from Arabic; something like a king with many women.
And by dictionary it means one man associate with many women.
With or without affection is not really the requirement for the word harem.
But then again, I'm just saying the word harem is legit for this manga by the root of its definition and not the cultural references.
If I were to look at this manga direction, the kids obviously likes him.
There is some affection too but not in romantic way, except for the water spirit (if you read the following chapters).
Maybe someone who have read the LN can pointed it out but literally, the harem tag is not wrong unless mangadex have different rules. On some websites they even put tag lolicons, which MC is not.