It's not about them forgetting but him being a perv casts doubt on his story and also it's his workplace and they don't really know the husband so most of the blow-back will be against him anyway. Not to mention the level of proof they have of abuse is the level that's regularly overlooked IRL...