So Brimhats are all about selfishness. I get it. Fulfil your dreams, punish everyone else and call it justice.
I see some Star Wars parallels here. Brimhats represent the Sith in their strive for strength for.. what ever is that Siths reason and Pointy Hats represent the Jedi, law and order and really bastardy like, cold blooded decisions to preserve the "greater good". I can imaging how they started. After many fuck ups by many societies the pointy hats rebelled and somehow those terrorists won. You can't just call someone a criminal, because they have been declared so by an unjust society, can you?
They outlawed certain magic and prevented unfit people from using magic. But my guess is that the list of outlawed magic was much shorter back in the days and that you could become a "fit" person by applying for a licence, like a drivers licence and a university degree. Centuries later this freedom fighter and hippy group turned into the theocratic, bureaucratic mess we've seen so far, that protects itself and all evildoers within it, by shrouding itself in mystery. Worst part is, that I believe they are still outlawing more magic. Sure. Nations abused it, so forbid every form of advanced technology.