Honestly, I'm just reading this manga version to see the scenes I've read about illustrated in a hilarious way (the art is great that way). If you want a more total grip of the plot, I strongly recommend reading the light novel, from what I'm hearing of the anime, the light novel is definitely the best version of the story. It has an official English release you can buy and if you'd rather pirate it, I suppose you'll have to go looking on yourself as I can't help with that since I bought it to support the author.
Don't bother with the webnovel version, by the way, it's simply not as good as the light novel even before we discount how bad much of its fan translation is (there're whole parts that are machine translated).
I suppose you could watch the anime instead, but who knows if it'll actually get to go to the end of the story. If it doesn't do well enough, it might not even get a second season.
Also, the side story stuff coming off as cliche isekai stuff doesn't last forever. At some point more interesting stuff start happening and that's all I'll say about it.