My guess for what’s happening:
In order to defeat the other saints, he had to travel alone to another world that has a dungeon harboring a special relic. No clue why the author’s practically ditching the other characters, though hopefully this arc doesn’t drag on.