@ScrotalLingerie I don't think this is even a "Harem" according to this sites definition, which states explicitly "romantic interest" as a criterion. The main character literally brainwashes the creatures he caught, whether they are female, male or whatever sex there might be in another world aside these two.
This means that there is absolutely no base to start any romantic relationship( as they are brainwashed to find in themselves no resistance against what their master says and does), otherwise hypnosis and/or mind break or inducing mental afflictions (e.g. the infamous Stockholm syndrome) would also count towards "romance" and "harem".
Of course, this is told in a lighthearted manner, but the truth on the matter is a pretty dark premise of near-perfection of the art of slavery.