Amongst many male social circles, they tend to exaggerate their "sexual experiences" and ostracize those that don't have anything to contribute to the discussion, leading to "virgin" being used in a derogatory manner. Those, often teenage, boys then get all caught up in their head and feelings...