@CaesarTG They really couldn't have; it was mentioned that the only time he saw the outside of his cell was when his older brother used him as a target for magic practice. While his particular brand of magic doesn't require any instruction, training, and such, that's a vastly different case for swordsmanship. You can
technically learn swordsmanship from a book all on your own, but all you'll know is the forms, the foundations, but you'll lack any kind of knowledge on how to
apply those things; worse, no book learning can really teach how to perform in measure, so even if you
could learn to apply your study, you would be left with an utterly rote and stilted form of swordsmanship that, honestly, would be worse than just flailing randomly.
The isekai aspect is really just a convenient way to have the protagonist have access to knowledge that they could not naturally gain, in this case, swordsmanship.