I've got a sad theory.
Obviously he can touch ghosts, we've seen him do it before, the sensei can too, she was literally fondling her boobs right in this chapter. I think the reason he can't touch her specifically is because she subconciously rejects being touched by him, likely because he's a man. So I think we can guess how she died and therefore why she chooses to keep that memory locked away.
