Go back to the icy lake, Kelpy.
@jennydude- So, what should he be doing instead, huh? As I see it, he's doing everything he can. If the mother was a mistress/concubine, it's entirely possible she wanted to live in a separate house. And that separate house was a mansion, no less, so it's hard to claim the emperor would have treated her poorly. However, the moment she died, everybody immediately forgot she ever existed. So, all servants in that house would have also been left wondering what on Earth they were doing there. Eventually they would have been relocated.