So I think the implication is that she did get raped but, as per her memory loss, forgot about it and the older brother found out about it because he was doing the same thing to other girls so was frequenting the same sites as the pig mask rapist. Judging from Yuu's face when she said it didn't hurt he probably knows she was already fucked but couldn't remember. And based on what the shitty older brother said about girls always remembering the guy they fell in love with that's pretty much the reason she remembers her time with Yuu and what he said even without something to remind her.
I think...