I feel a little bit bad for this guy. He seems to be a bit too much of an "herbivore" to handle these sorts of shenanigans.
Also, I get the sense that maybe polygamy isn't quite so accepted in this world? It'd be an interesting change of pace for these Isekai manga, though it would mean the others wind up getting left in the cold. I'd especially feel bad for Hi-chan because she put in so much effort to be with the MC. She's just such an honest little lady... I won't quite say "innocent" because that bit about her wanting an egg, but even so...