You know, it might be interesting if there is a legitimate magical/cosmic force preventing him from getting freaky with the summoned girls. It'd get some laughs if the girls find out after they start wanting to be touched that he actually can't touch them.
I don't think it was a magic restriction that prevented him from touching her,
more like the Gentlemens' code of ethics that kept him from pawwing at her when she had her defenses down.