Based off the first chapter, it seems like his party actually did realize how good the MC is. They just didn't expect him to grow a backbone and leave the party, which will make them lose the only frontliner that could deal with their crap while fighting