I've honestly never really understood the whole "I'm upset that you spend too much time around other people that are of the same gender as me when we're dating" kind of complaint. It just seems baffling to me to want to control who your partner associates with in that way. If you felt they were that weak-willed, that likely to cheat, why are you dating them?
I do appreciate the thought here that it's probably not actually for that reason that she said that and the next step is to try to see if there's something more going on. I do also hope that at some point Subaru is able to better work through his own anxieties about Kasane spending time with men.