Here's my theory:
From what we've seen he has only transported between worlds when he sleeps. So it could be that his soul is still tethered to his world and Mary used a spell to summon him to her world, but since his soul is "foreign" he will be pulled back out naturally if he is left...