Speaking a language must mean he has blood from one of those races? Is it inconceivable to learn another language? I mean, even Japanese learn English. Sort of.
Cultural thing perhaps. People on Earth have better lines of communication, so learning languages from other places isn't as hard. But even the Japanese had considerable difficulty learning English as recently as mid-twentieth century; case in point being their declaration of war which arrived late. Here, the beast people are treated as menial laborers, and until the MC, were treated as among the lowest of the low by humans, so most probably wouldn't even bother to try learning.
Or even physical mouth articulation might also be a concern. Humans might not be able to make certain sounds which make up the language, either because it's not physically possible, or lack of proficiency, because no one who's ever tried before has ever succeeded. Which might also imply the issue of the class and racial discriminations.
And they were talking about Hinoharu nailing local accents and speaking like a native, so if we take these things into account, the only way he might have learned these languages by their reckoning, if these assumptions are correct, would be from having local background.