Yeah... Nanjou-senpai is pretty damn pushy, but Yamada does need to make it clear to him she's not interested at all... though that 'sayonara' certainly was pretty blunt, based on the TN.
I also agree with those that say he stopped Kyo from prematurely confessing right here. He kinda realizes the situation a bit better than most, even though he's tried to use some sneaky and subversive tactics just to get contact info.
Kyo needs to confess with his own courage and straight to Anna, and Nanjou understands this. Kyo still has some more growing to do before he's truly ready for that moment.
Little by little, he's working through things for himself. This story is just as much about his character growth, and Anna's, as it is their relationship.