MC at all times has many babes actively seeking him
What? Did we read the wrong manga perhaps?
MC only have his two established girls, with the knight being the third and last one. Through the whole chapters + RAWs, which are more then 22 chapters, the overwhelming major focus of sexual content is not MC, but random/other male fucking various of girls. This manga may had harem elements in the very beginning, but looking at it objectivly (and knowing what the author fetish are when you check yourself on nhentai or hitomli), its kinda hard to ignore the statement I or he made.
Further, it dosent help that such scenes exist which im pretty sure is for a small audience but wont be liked by those who want a harem. And according to the current raws I saw, THIS is the focus and not the MC being actively seeked by various chicks. In the end, the logic here is as long the three main chicks remain untouched and have some scene here and there with the MC, its somehow still a harem when the overwhelming focus is voyurism or rape scene of other female side characters.