Oh man, I remembering remembering this series and trying to find it. I think I had looked long enough that I started to convince myself it was a figment of my imagination.
I do absolutely adore the start of this story. In most isekai, the protagonist finds themselves in an incredible new world with all sorts of opportunities, skills, and friends/lovers. This one hammers down on "you're in a new place with nothing, you have nothing, you understand nothing, and you are so close to being nothing". But it's not edgy, like those manga with antiheros who get backstabbed by their BFFs and swear eternal revenge - it's more of a real feeling approach, that the world does not love you and will not cater to your needs. I love it.