At this point nobles and thug should've heard about the rumors that there is a powerful mage living in this town and they should be careful not to mess with anyone or else they might catch his attention
Nothing to do with the teleportation. He is holding the dragon marble he got from the dwarfs. I think he is absorbing it (or vice versa) and his mana is fucked during the process. So he is practicely depleted at the moment.
I think it's a combo of both. When the transportation spell was introduced, it basically took all his mana, even though he's a 9th level mage.
Combine that with the marble thing, which might've left him with just enough or less than the amount of mana needed for the spell. Yeah he's drained to heck