Not really the biggest fan of harems, personally, but I think this is one of those rare cases where it's probably fine if he gets with both. I don't think it'll affect the story in a major way, so if the author makes them end up together, I'm ok with it.