The consequences of owning a harem have run its course, it seems. Those other girls may not even be seeking love and affection from the emperor, either, just enough time with him to gain some favor for them to gloat at the other girls, or if they have some decency, have political talks to help the countries they come from prosper. Very shallow and cold, respectively, but expected in political marriages.