Let me guess, the people that MC believes to be his descendant, aren't actually his, but someone who took over his family name and heritage (many generation back).
On the other hand, the baby prince and MC's mother might be the real descendants, and MC mother begged that idiot noble to father a child with her (which results in MC being born), might have something to do with a prophecy, and MC's mother wanted her child to get the stolen family name and heritage.