I don't think he's had trouble making friends. Chapter 1 page 8 has him mentioning that he had fun with his friends, and didn't realize how precious the relationships were until he was translocated. Which makes me think he has a harder time establishing close relationships.
Growing up in an orphanage (the "facility" as mentioned in the manga), I'm sure he had a rotating group of caretakers and was never treated any different from any of the other orphans. So he likely has minimal experience with forming close bonds with one or two special people, especially after adopting his current "be kind and polite to everyone" mindset. He had friends he hung out with, but I bet he never had a best friend, his one true childhood friend.
It should also be noted that he never mentioned asking a girl out. By his own admission, he went out with girls who confessed to him. Surely, they saw his kind nature and wanted to get closer, only to find out that he doesn't know how to do that. He never learned. And, knowing that other nice guys who can treat someone extra special actually exist, they broke up with Masato...
...But the girls in this world know they don't have that luxury. For them, Masato is the one who's special. Koumi even struggles with this way back in the second chapter. She also wants Masato to treat her special, like his ex-girlfriends from his old world. Only, she doesn't have the luxury of selection, and she knows this. So she wrestles between her jealousy of wanting to be Masato's only, but still willing to do anything just to still be close to him.
The other girls are presumably the same, which leads to Masato's statements about his own nievety in the first chapter. He thought he could survive just fine in this new world, and to his credit, I'm sure he could "survive" just fine with this attitude in his old world. Only, the girls in this world won't cut and run. They're going to get more and more possessive and desperate, and his way of thinking will be forced to adapt... one way or another.