Mnnn strange in the novel he thought that they were a mere concubine and concubine daughter that is why he was angry for their behavior(cause in ancient times they should not be so impudent) not stepmother and stepsister .
They skipped the part when he started to gradually receive memories that is why he could adapt better