Worries because he also knows what kind of person Ryouma is.
Ryouma wouldn’t listen to things that’s not like the “reality” he expected.
A big worry in Kotaro is, if Ryouma keep doing his approach, one day Shiho would be too exhausted, stopped thinking and just fall to his influence.
Ryouma's main characteristic is he is too confident. When facing that kind, young person with not enough experience might unconsciously doubt themselves. If this dude is so confident, maybe is is right after all. And because he will not listen, you'd run out of arguments trying to explain it too and became unsure, maybe it's you who is wrong after all. Also, usually people would try not to be confrontative, so they try to properly explain, try not to have prejudice, but this is against someone who wouldn't listen.
It's also infectious. Ryouma is the type who would do it in front of public, and that might change the opinion from around into something like a peer pressure to Shiho, like she is wrong for keep refusing (even if they don't say it directly). She might even fold just to escape from it all.
Even if she get out this time, that guy wouldn't give up until he gets what he wants, and in closed community like school, it's difficult to just stay away.
Well, it’s also like the usual romcom story pattern, the heroine starts from hating the protagonist, but as they interact more, she saw more sides of him, begin to get interested and want to know better, until the feeling became uncontrollable.
Worst case in this one, she might start trying to convince herself there are reasons for her to be in love with him after all, to believe she is happy, to escape from the suffering.