He started living and working in a different city before the affair started, so it's not like he's there to see anything. Also, she met Hinata less than half a year ago, so the affair has been shorter than that, probably only a month or two. He also doesn't seem super interested in reaching out...