That concern could be resolved quite easily if Shinobu simply explained the real reason he doesn't want Fumi to take the role. The panel showing how uneasy he felt watching Fumi being physically intimate with someone else during the performance already makes his feelings pretty clear.
And if he overheard Fumi mentioning Sakura, I think that would make him even more likely to give Fumi a clear answer. He wouldn't want to repeat what happened with Sakura, when he assumed she would understand everything once they had the chance to talk things through, only for her to leave with nothing more than a message on LINE.