The "Ever Increasing Wives" part of the title doesn't seem very fitting. Sure, he's got three women that are into him (four if you count Rin, but I think Rin is just affectionate by nature), but it's nothing like most harem isekai where they add a new girl or two every volume.
I'm not complaining, mind you. Adding new characters all the time is usually a sign the author has run out of ideas.