I hesitate to call this sidetracking because this is important to the overarching lore of the story and the past of both our characters, that's also why the previous side stories existed so in theory I think this detour is good.
However the problem lies in the execution, namely that unlike the previous flashback stories that were more emotional affairs with some sprinkling of lore, this one has way more exposition in it which makes it hard to get invested in the romance at the centre, and said romance was basically just instalove (even if we know that white mage dude screws him over by the end) with sex constantly being mentioned in a way no other romance in the story has gotten. It's all very old yaoi, which is a bit befuddling because otherwise the manga has been much better at justifying people's deep emotions.