I swear, I thought I'd responded to your message before... So, we are indeed missing context as readers that we will eventually get in a future volume. But, the relationship between Kaibara and his wife was not nearly so bad as Yamaoka perceived it. I would add more, but it will be covered in a later volume. Of course, Kurita, Tomii, and Tanimura (who were there when he blamed him for killing his mother) won't have much context either, but given their personalities (or at least Kurita's and Tanimura's), I think they probably feel like he's exaggerating and that if they would sit down and talk they could work things out. Toyama certainly has the full context, which is likely why he's the one that pushes hardest for reconciliation.