This manga is a bit unconventional. It basically seems that it's taking a DnD campaign with maybe a few homebrew elements and turning it into a story.
I think the low rating is due to some people just really not liking the DnD aspect of things, since they feel it detracts from the story (which is fair).
However, I think that if you're willing to think of this in terms of a tabletop game instead of a truly realistic narrative, it's pretty entertaining. At the very least, this story is unique amongst the 9001 generic isekai clones (not hating on isekai, I'm true isekai trash, but a lot of them set a very low bar in terms of originality and quality).