Honestly I’m baffled by how this works.
They detest him for having low magic capacity, but the uncle states they have no way of measuring magic.
Furthermore he has the potential to become the best if he understands it enough so even the capacity isn’t a limiter.
Basically he’s detested for an ability he could become a recognised master in. Makes no sense, and that’s before you add on how the uncle is recognised by the family despite being in the same boat.