For once I can get behind this "excuse". For the cafe to function, he needs everyone working together. It already took him a lot of time and effort to get along with the girls. If he started dating one of them now, blowing up the status quo, he risks coming into conflict with not just the other girls, but causing a rift between the girls themselves as well. That's not a risk he's prepared to take at this point.