So what he thinks Felix just had a giant growth spurt, and all the documents and records around him are forged meticulously? Even if he is that paranoid, there's no one around to rein him in besides Felix? Not even the court physician? This is ridiculous, the emperor is naked.
Suspension of disbelief only goes so far.