So, it wasn't completely clear (probably because the duke himself has to avoid clarifying his mistake to Aisha) but did anyone else get the impression that the duke believes Aisha is Abel's birth mother? He picked him up on a battlefield, so he doesn't know the mother. Then out of nowhere this woman appears who is going out of her way to shower Abel with gifts and loves him unconditionally. And she is being supported by some other third party who is also willing to go to extreme lengths to keep her and Abel together.
So if I were the Duke, I'd believe she is the mother and 'Stanley' is the father, they had an illegitimate child they had to abandon, and found that the duke had picked him up. They tried to support Abel anonimously, then the duke misunderstood and made his offer of marriage, and so on. So he's warning her that he loves Abel as his own and won't give him up, but he will let her stay by his side as his wife.