Feel bad for Elle, can only imagine the amount of blueballing she goes through when he's practically boinking every other woman in his life.
kinda feel like she is the endgame woman; its the woman he has invested the most in, and in terms of magic ability and intelligence, she is slowly ascending to his level. Still a ways away, but its getting there.
The biggest thing between them is his former world's idealogy of research partners having no intimacy. He said he would not limit himself by those rules any longer, but Elle strangely became more reserved after that, focusing on self-improvement. So yeah, all that to say I expect their relationship to be the last main one to be built, once Elle has become more Yard's equal.