Ironically enough, this story could have worked if the author had played the trope straight.
Instead of focusing on a bunch of hypocrites who could all have saved the banished guy but chose not to for no real valid reason and then refused to fix the misunderstandings solely because the author wants to get that Daru x Athena ship sailing, the author should have made them all be dickheads on purpose and simply played the same average "banished from the party" straight, just from the party's perspective.
Imagine if we had been accompanying these idiots comically failing in their adventures time and time again, slowly coming to the realization that they kicked the guy that was carrying the party, eventually accepting that they screwed themselves over, and when they go invite the guy back to the party, he's either too powerful for them, found a better party or has died trying to prove his worth alone in the dungeon.
With the first half of the story showing how dependent they all were on the guy they banished, and the second half being about the party on the brick of disbanding because they have lost the guy forever, and then having to figure out how they will handle the situation going forward now that everyone has to carry their own weight and properly learn to synergize with each other, I'm sure we could have got a pretty good story!
The main issue here is that the author tried to put the antagonists on the seat of the protagonist, and in order to do that, he had to give each member virtues attributed to heroes (never give up, always think of your friends, is actually secretly strong, training arc, etc.), and ended up giving Est villainous attribute (mocks those bellow him, treats women unfairly, gloats to others around him, etc.) to make them look better in comparison. If he had made the main characters villainous and Est heroic, but the story played from the perspective of the antagonists, this could have been gold!
But alas, the author was not self aware enough to have a good idea beyond an interesting premise.