@G-P
Well he had no idea.. and neither did she. She was a virgin when they got married.
However, it doesn't fucking matter if he only wants sex once a week or once a month, cheating is not OK. If she can't handle not having sex, then it's her responsibility to tell him that and divorce him (or he could, I suppose, let himself be cuckolded in the actual definition of the word). The wife is a complete scumbag and as
@boag said, is just in denial about not having cheated. The moment she kept in contact with the young man while having lustful thoughts is when she cheated. A good person would immediately remove themselves from the situation at that point and would no longer associate with him until free of the contractual and moral obligations of marriage and a monogamous relationship.
That said, it's a fuckin hot manga and it's not real life, and the audience's point of view is mainly from that of the young man, not the older husband. If the point of view was from that of the husband, this would just be NTR and we'd all be hot, stinky garbage for reading it. But since our POV is from the buck, not the cuck, we're just... I dunno, cardboard recyclables?
edit: I should note that in the interests of a full disclosure, I have slept with two married women (while their husbands were at work) and it was some of the hottest sex I've ever had in my life. It was also a complete dick move by me in my younger days and I wouldn't do it again.