"He named you two himself." Uhh, they don't like him? How is that a point in his favor?
I'm an adult, and I wouldn't trust the royalty either. With the exception of maybe Alex, they run the gamut from psychopath to incompetent. No matter how good everyone claims the Emperor's intentions are, the results of his actions are much more relevant, not to mention measurable.
The way the Emperor runs the place, just throwing money and riches at his problems without any care or consideration, I'd want to get out of the way. People with that little sense ought not to be given so much power.