I feel like he's too paranoid about being recognised. Unless magic body possession is an epidemic in that country, why would he care so much? He would more likely appear to be an amnesiac or just merely look really similar to someone else than to get into real trouble. Unless, of course, the previous owner of the body was a wanted man with a huge bounty on his head. Bounty hunters wouldn't exactly care to listen to his explanations.