That's typically how it is in these series that sort magic by element/colour. The blank/colourless/attributess magic is always seen as either completely useless, very basic spells that wouldn't be useful outside of slight QoL improvements or non-existent. It's wild to think they'd get so ostracized that they couldn't even think of possible other ways to use their magic until someone with outside world knowledge arrives. Societal bully be powerful in them fantasy/medival settings. 💀
MC was left to die as a infant, so is possible that peasants just kill colorless on birth.
But is weird as the noble family treated MC sister the same as MC, I can understand (not accept) that the family and the servants thought it was fine to let a colorless die, but what about the girl, if it wasn't for him she would have died.
I am of the opinion that in the original story after MC death for abandon the servants were fired and when the villainess was born the new staff spoiled her rotten, but as MC survived the incompetent stayed, and seeing as a inferior infant survived that treatment they ignored the new one, lazy nobles, lazy workers.