First off in what universe does a so called "main family" always have the best potential? They may have the best initial talent because they can afford more resources at the beginning, but someone's potential is pretty much based on half luck and half personality. I also feel like the mc is just gonna be like really slow development at the very beginning of his training and peoples expectations will be continuously lowered until he just skyrockets like overnight or something.