Neither men nor women evolved to be monogamous, but casual sex (specially in women) affects pair-bonding (the ability to develop empathy and affection towards a partner, it's got to do with oxytocin). For example, studies have shown direct correlation between a woman's sexual partner count and the (increased) probability that she will initiate divorce.
Men aren't as affected, mainly because men's reproductive strategy is having unlimited access to women to spread their seed, whereas women's is hypergamy (or monkey-branching to higher value/status males).
Add other reasons, like women's hidden estrus, and the inability of men to prove paternity, and you get men preferring virgins/low notch (sexual partners) count.
Marriage used to mean something because both parties involved needed to sacrifice their respective reproductive strategies in order to raise a family, with men providing resources (because evolution makes them better at this in all aspects, like being 2/3 times as physically strong as women)) and women giving birth and taking care of babies (obviously because men can't become pregnant and they get a crazy cocktail of hormones that allows them to bond and prioritize their babies).
So no, men preferring women that are virgins/have low N of sexual partners isn't misogynist. It's nature. It's evolution. Calling somebody an incel or a "something-ist" doesn't cut it when men are biologically programmed to be like this, just like women are programmed to use their most important assets (youth, fertility and beauty, as shown by multiple studies) in order to attract better mates.