... So he believes in the past life he remembers, but completely rejects the idea that it might have happened more than once without his memories carrying over from the previous ones?
I mean for all we know, what if Yahiro and Ketouin really were together in previous lives BEFORE the one he remembers, and it is hurting her inside to see that he only remembers the one with Hina?... Imagine they were a fated couple but he lost his memories in his previous incarnation and she couldn't find him... and now she has to watch him chasing after Hina instead?