I think more than anything else the way he ignored Ruka as she was chumming it up with his grandma to answer his phone told Ruka that he only has eyes for Chizuru.
Ruka blackmailed him and their relationship is based on that heart-rate gimmick that happened spontaneously. I don't get why some people think like the guy owes her something.
He goes on dates (well, fake dates, ok) with her, they live next to each other, go to the same school, and yet he hasn't the courage to talk to her on the phone? For real?
I understand that Kazuya likes Mizuhara more but your family seems to be opening to the idea of Ruka and she isnt payed to like you its time to jump ship instead of comlaining about how it is a bad situation