I also think that the baby was concubine-kun's.
We can theorize it as -> affair between queen-chan and concubine-kun's. Then king-san loves concubine-kun but hated queen-chan cuz of their affair. Then king-kun takes every opportunity to get rid of queen-chan. Baby-pyon looks like concubine-kun...