The problem is almost certainly that Monchi02 (the writer) had an interesting idea with which to start a story, but no idea whither to go from there. And nothing has since come to him, so he's been churning-out a very generic story. Now one of those generic elements is dissonant with the initial premise.
Very much this. There are two ways I think this
could have gone: the Est Dies route and the Est Lives route.
In the Est Dies route, we get the same beginning, where they kick him out by saying horrible things to him and Est goes to the dungeon out of frustration to try and prove he's not worthless. Only, he actually dies, because he doesn't get any plot armor and he was always
just a buffer. Stardust learns of this and they're positively
crushed by guilt, Athena in particular since they're childhood friends.
From there, it could also go two ways. For a more tragic ending (for a loose interpretation of the word), it could be them figuring out that Est was the only thing carrying them and after a string of failed missions, they drift apart, the party breaks up, and everyone goes home. The end.
For a more bittersweet story, they could learn that Est was carrying them and struggle improve, slowly regaining the strength they thought they had, while always carrying the memory that it was only because of Est that they had such an easy time of it and, after awhile, they retire as adventurer's and open an adventurer academy (or something similar) dedicated to teaching and training new adventurers so there won't be any more Ests.
In the Est Lives route, we get the same flow of events up until the dungeon. This time, it's
truly an "expelled from the party, but from the perspective of the banishers" series, where we see that they truly regretted expelling Est, but thought it was the best for him. Then they learn he went into a dungeon and not only survived, but unlocked some hidden ability to allow him to thrive, but Est never actually returns to Stardust at this point.
From there, Stardust just gets weaker while nobody can stop talking about Est's meteoric rise, rubbing their guilt in their faces, but we don't actually
see Est until Stardust has actually started to claw their way back to the top once more, this time under their own powers (sorta like in Est Dies bittersweet route), just as a three-man group. Or maybe as a four-man group with an older woman as their fourth who becomes a proper love interest for Daru.
When Est finally
does reappear, it's well after everyone's had time to calm down and reflect, so things are a
lot more chill. Stardust apologizes for their harsh words and explains why they did it, which Est accepts, saying he'd already figured that was the case and didn't bear them any grudges (any longer, at least).
The best ending for this route, I think, even if somewhat generic, would be if Stardust and Est and his new party all joined into a single guild, Est and Athena got together, and they all contributed to starting the adventurer academy from the bittersweet Est Dies route.
If nothing else (and I'm fully aware I'm tooting my own horn), any of these options would probably be better than what we've got now.