Btw the whole idea that it's 'easier' or 'better' to learn languages early has 'recently' been somewhat discredited. It doesn't seem to be entirely 'wrong' but the truth behind it is far from the simplistic way most understand it.
It can be a aspect to the speed to picking it up, how easily the language can be more 'deeply ingrained' into one's thoughts, etc, but there are many more variables that seem to be much more 'important' influences overall.
I can probably dig out a few of the papers on the subject, but I don't have them at hand or anything.
And obviously we are still fighting to understand so much of the brain and everything about it so who knows what we'll figure out in more time.
As for the argument about him learning the language. There have been valid points for both perspectives, but aside from those things already mentioned, it could just be something like a 'magic' reason he just can't actually learn it, even if he wanted to and everyone put effort in. Like a 'mistake' in the summon, or an inherent limitation, a 'curse' or something, etc. Yes, not exactly a 'satisfying' reason, but it's quick, easy, maintains the premise, and keeps the many possibilities of holes that could arise from some of the other explanations somewhat more limited.
Maybe this was already mentioned, I didn't go through all the earliest comments.