Let's look at it this way:
From Yuu's perspective, he invited her to go somewhere, but she said nah, let's stay here.
From Yami's perspective, we see her own thoughts where she goes
She doesn't think "bah, he isn't inviting me anywhere specific", she literally thinks "I want to be here with...