I went back to the first chapters and i see only a faceless woman with longer hair as Iruma's mother...
And there is also still a theory that he could be adopted too.
Again until we have not explicit stated, that she is not related to him, it is all a THEORY.
That is the fun part to read something, making theories with the information you have.