Once you start throwing magic into the mix a class based society starts making more sense because of how different it is from our world. For example in ascendance of a bookworm only Nobles are capable of wielding mana and if a commoner is born with Mana it's slowly kills them. And Mana is absolutely necessary in enriching the Earth to allow crops to grow and to keep the water clean and the Animals healthy. If Nobles don't regularly enrich the Earth with their Mana it's slowly turns into a desert.
I'd argue that this particular kind of class based society doesn't make sense because there's too little upward/downward social mobility.
To give an example, let's say that A is the mediocre eldest son of a Duke, B is A's extremely talented illegitimate younger brother, and C is a similarly talented random commoner.
In the real world, the benefits that A gets from stuff like being the eldest son, having a powerful mom, being the default heir, etc., are enough that he can be significantly more powerful that B and definitely C.
In a world with magic B and C could potentially have magic powers that are useful enough to outweigh A's advantages.
If you don't adjust the system so that B can potentially inherit the Dukedom and C can potentially earn a title the Bs and Cs of the world are eventually going to rebel and tear the system apart.