The reason the Emperor says "Children are difficult" is obvious. There are no normal children in the direct pure blood lineage of the Empire. by 4-5 they are already a teenager guaranteed. There are no "children" in the Imperial family by that age as they will always be exceptional geniuses who learn 3x as fast as a normal person. These two are basically the first "children" that the Royal Family has ever come in contact with (In spite of the fact that their "mental age" should be 25. At this point they've clearly regressed mentally)