Nothing. It's possible to do whatever, ignore your class and force anything you want, if you're ready to put in the effort.
But sticking with your class makes acquisition of skills easier and more straightforward.
Edit: Also, "Manabananaman" sounds way too cool, so that's a no. But overall your thinking is right.