I will never understand this particular idea and set-up. Adventuring parties keep climbing by surviving, being conservative, and minimizing risk. Yet here you have a guy who can turn nearly any encounter into a talking encounter. Can you imagine the number of random 'oops, stumbled across this pack of trolls' grindy potion consuming bothers that just get turned off by homie being able to talk it out? The number of additional eyes to make perception/insight rolls in a huge ass forest? Wild Empathy is busted and dude is on waaaaay more powerful of an effect. I can't imagine playing with a party that doesn't want the ability to pick and choose its fights advantageously.
There are actually many cases where it makes sense.
For example, the manga where fights are more a form of entertainment rather than actual conflicts. And the MC uses dark magic (mostly debuffs) that is extremely efficient but doesn't show on screen. That makes for bad entertainment value, so the party needed a more flashy fighter to increase their popularity.
Or the case where levels are quantified and the MC stagnated at level 1 for years, without any special skill to speak of either. The fact that he did end up awakening a special skill that finally lets him level up after the eviction doesn't retroactively invalidate the reason for said eviction. Objectively speaking, this guy wasn't cut for fighting and the eviction was justified. (Well, kind of. In this manga, a lot happened and he was treated excessively unfairly.)
The problem are the times when the MC is expelled because he has an invaluable support skill (like here) that just doesn't match the brute force mentality of the party. Generally, the party realizes a bit late that their brute force approach to everything never really worked without the supporter's behind-the-scene preparations or corrections.
Examples are tamer, support or healer MCs. (Healers in particular are strangely common victims of this trope for some reason.)
Edit: Note that this is also often coupled with the MC's inferiority complex, which means he never bothered explaining how he helped in the various situations that they would never have survived without his contribution. This seems awfully frequent in manga, so I'm always happy to find new stories where the MC is not crippled by an inferiority complex.