Seems like a bullshit twist, MC was sealed in a tower and never left, he used his hearing to learn chants used in his range, but he wasn't going around using them, is a completely illogical way to twist the conversation, the elf had no reason to connect MC to that guy, the father even thinking they could be the same person for one second makes even less sense, I think the author just wanted to bring up that name as someone going around stealing magic secrets.
Even the reports of MC helping the princess doesn't make sense as the witness (guards) were beaten black and blue and left unconscious, and MC is a unknown dude, again, incredibly bullshit twist, unless the manga adaptation cut on a lot of stuff and in the novel MC destroyed half the castle when leaving, left the kidnappers healthy and proclaimed his identity to them.