@tseng i think you got the wrong picture to explain the situation in the next chapter.
First, no, he's not being pushed away by the girls, that's just all in his mind, he is afraid that he can't hang out together again because now Shikimori become popular among the girls.
Second, also no, the girls is just surprised that's all, i don't how you get the conclusion of "the girls dont give a shit and dont let her move" and Shikimori can't push away through the crowded of girls who surrounding her.