@CBC As you mention, the MC does have trust issues. Very, very, intense ones, considering he was contemplating suicide before being isekai'ed due to being ostracized and bullied.
The only reason was able to interact with the first people he met was because he still treated them like NPCs at the time.
That means Roxanne was the first person he started to be able to interact normally with. Even so, notice that he's been abnormally and secretively wary of Roxanne's feelings even before Sherry arrived. With both Sherry and Roxanne he's obsessed with being weighed "positively". They are the first people he's probably had a "normal" relationship...maybe since ever.
The reason I explain this is because he's not so motivated by logic, fear of death or compassion, as he is by his past trauma. For example, it took him talking to an adventurer who died immediately after giving him his story to finally seriously contemplate the idea of dying.
But that's fine, because that's what this story is really about. Not so much about a harem protag who is super careful and does things in a logical and clever manner. But a guy who is slowly rehabilitated by his own slaves, overcomes his past trauma and becomes a better person for it. That is why the grindy, windy pace at which the story develops by the way that many complain about. Read back knowing this and you'll see loads of examples of how he's slowly being nudged and prodded on this path in almost every chapter even in the manga version, and it's pretty methodical.