It's interesting to see how all these stories are interconnected. For example, we can trace this tragedy back to the death of Lady Aadou's (forget the spelling) son.
Rakan had gained his position through his uncle, Mao Mao's foster father. At the time that he slept with her, Lady Aadou gave birth, had the complications that left her barren, possibly swapped her child with the previous emperor, and her child died (because of the honey the head made fed him). This led to Rakan's uncle being punished (either for not looking after Lady Aadou properly, or not noticing the potential baby swap). But as we know from previous chapters, when someone is punished for a grave enough crime, their connections are also punished. So Rakan himself was sent on an expedition at the same time.
He thought it might take 6 months, which if so he could have returned in time, but it took him three years. Not sure if they received his initial letter, or why he never thought to send another letter during those three years when his initial plan fell through. At least to hear back or make assurances. Maybe all the letters were intercepted? This caused the misunderstanding that he had slept with her and then ghosted her, which caused everyone in the brothel to hate him.
Here things get a bit tricky, because while yes, three years is a long time, Mao Mao would have only been two years old at this point. He went back to the brothel, which had been going through hard times as well after their most famous courtesan lost her status (possibly why the granny didn't vet clients properly and had Fon Shen catch syphilis, further ruining them), but apparently granny refused to listen or believe him.
The thing I really don't understand is why Mao Mao's foster father didn't help. He would have known Rakan visited often. He was eventually trusted enough to adopt Mao Mao, and obviously recognized their connection (why else adopt her). He would have known it was his fault that his nephew was punished and couldn't return to redeem her. Instead he adopted his nephew's daughter and refused to let him know about it, or assist in clearing all the misunderstandings.