The MC is doing the correct things here.
The kingdom clearly summoned him to use as an enslaved/cursed human weapon to conquer othe countries.
If he leaves witnesses who got a good look at him or heard his voice clearly then they could track him down easier.
The young doctor told the MC where to go and could have relayed this to the guards.
And for anyone who is saying he should have "knocked out" the doctors...
Humans don't have off switches.
If you try to knock out someone then you'll likely kill them instead and tying someone would take too much time, every second matters when escaping a castle full of people who know magic