Y'know, as tropey as it is, at least the reason he got booted from his first job makes sense.
If you're going to watch gladiators fight or whatever, that's a gimmick that's only cool the first couple times, then it just gets boring as hell.
Maybe it was the only place that would hire him at first, but as soon as he started winning fights and made an impression he shoulda tried to switch to being a mercenary or something instead of staying in the career where constantly beating everyone in the most uninteresting way possible is gonna make the audience hate you.
Then he traveled to get away from his bad reputation there, but once you're in a new place it's not like anyone has any reason to think you're special and the people going by the "well he has no class so he must be useless" makes sense - he hasn't done anything that would disprove that, and yeah it's because they won't give him a chance, but why should they unless they're a weirdo princess that's willing to challenge stinky strangers to fights?
It is a tired and overused trope but it's also nice to see when usually shitty tropes are executed well.