Ooooo here's a thought. Remember that line that said she was wearing that hairpin before she went back in time? What if it was Yulan who enchanted the hairpin with some kind of time reversal magic to save her from her death? It would explain why he seemingly didn't do anything for the girl he was so deeply in love with when she was sent to prison and about to be sentenced to death, AND it would also explain why he remembers the first timeline since he was the one who reversed time. It never made sense to me why Yulan just let MC die in prison since in this timeline it's clear that he's been deeply in love with her from the start and would do anything for her, so supposing that he might have been the one who reversed time would definitely explain everything. By the by this isn't spoilers or anything just my own conjecture based on what we've seen so far