Everything is a chemical compound
@AngryFerret. Food, water, etc. are all compounds. I'd assume posion resistance only affect against things that would negatively impact his body in the sense of causing damage (ie stuff your liver wants to filter out).
If it just did it indiscriminately, potions wouldn't work and, more broadly, you wouldn't be able to drink water as it is a compound of hydrogen and two oxygen atoms and all foods are compounds of multiple substances. It's safe to assume that the aphrodisiac isn't anything more than medicine that would increase your libido (or that might make you sleepy) that your body naturally produces, which could bypass the filter completely. It's nothing that would cause damage to your body like a drug (as we have seen earlier) or alcohol if it's just going to release hormones, because not only do you do that on your own but certain foods will do that automatically.
It's also feasible it's distinguishing between harmful drugs vs nonharmful drugs, but that's a bit of a stretch