Or maybe, it has to do with genuine feeling.Could also be that yielding isn't just saying you yield. You have to mentally give up. Takakura just said he yielded but he was just trying to get them to go away.
Most of the time, when someone says "I Yield" to someone for knowing something, they just do it to remove the annoyance (because majority of people think of Trivia info as non-sense or annoying).
But in Takakura's case, he genuinely accepted that "yeah Germaine knew more than him", thus he didn't feel bad yielding to him.