I think he had to join other adventurer's parties a couple times when he was field walking to towns he hadn't been to before, like when he was heading to Palmasque. I think he might have also done it once or twice to get to get to a higher level of a dungeon with an explorer, maybe?