Well, I can understand why he wouldn't touch them. They've been drugged and aren't in control anymore, so having sex with them would be pretty bad.
What I don't understand is how he didn't have sex with some of them in normal conditions...
I mean, Yuuna, Sagiri and Chisaki clearly love him, even if they didn't really push for sexual relations yet, so OK...
BUT! Oboro has clearly stated and shown that she wants the D, Hibari clearly wants physical contact and Karura has offered more than once...
And we know that he loves them, and that he desires them too.
The only reason to not have sex with any of them yet is the true Harem ending with no jealousy (and an orgy ?).