Tbf for those dumb knights, this is one of consequence of Al's "Villain Act" backfiring, thanks to him putting all the blame that supposed to be on his dumb brother, and Royal Family onto himself (and Char's false accusation not cleared), most people get into belief that it's all Al's fault.
And for this case, Knights blame the lack of manpower on Char, cause "They have to protect this fragile crown prince's lover" something that to this point, we could say it's Leon's fault again, but because of Al's action, and how the new "rumor it's all Al's fault" isn't that spread yet, Char is still being blamed.
Hopefully, this Knight blaming Char incident would start making Al realize his mistakes.