This is one of my favorite isekai/harem stories and it's worth a reading, don't worry too much about the silly premise because the story manages to be a very entertaining power fantasy.
Before going any further, this story only has 21 chapters here on MangaDex, but a quick google research and you can find up to 40+ chapters translated on other sites. Ok, lets continue:
Despite MC's initial impressions, we quickly learn that he's very likeable and selfless character, he don't think twice before risking his life to help others, he's also very respectful and mindful of women and never pressure any of them into a relationship, but he also doesn't shy away whenever a woman shows genuine interest on him, he is level-headed and street-smart and can be ruthless with his enemies if necessary.
He's also very faithful to his first wife and don't plan on two-timing her so he reject the second wife at first because he's already in a relationship and also is wary about the idea of harem since it could lead to infighting between his wives and backstabbing, he only changes idea because his first wife likes the second wife a lot and is ok with polygamy as long as Lucas keep giving equal love and attention to them both, so she makes a plan along the sword to make Lucas get laid with the second wife.
The pacing of the story is very nice and the author don't rush events, he also slowly introduce new girls to the harem and takes time to develop them before going to next girl.