If they aren't in his strike zone, then they aren't. Besides, the "charm immunity" has also been played up as a running gag.
"Charm immunity" is the worst fucking trope, it just stagnates character and relationship growth. The majority of the girl's interactions have been based on their attraction to Zeff and the two paths forward are:
[1] Zeff responds in kind, developing romantic relationships with them.
[2] Directly and clearly turns them all down and have them get over their emotions and move on, whatever that entails.
Obviously option 2 isn't going to happen but I also don't see them moving forward either, it'll just be an endless repetition of current events because it can be hard to write genuine human interactions and this way the mangaka can also base all the jokes on Zeff being an old man with no interest in women.
And even if their relationships do start to develop
in literally any direction it's already waay past the appropriate point in the story for that, in the sense of everyone has been acting the same since their introduction for tens of chapters.