@blurgh123 In addition to what the others already said:
The father made it very clear, that he considers the MC to be his son and true part of the family. Therefore there probably never was a point where they talked about the time before he joined the family (he looks to be around 5-7 at that moment, so no big need to talk about stuff before)
And why his younger sister doesn`t remember: From when are your first memories? Mine start at around 4 years old, but those are mostly only pictures and impressions, only very few "stories" where I remember stuff that was said. The sister looks to be younger than one year old when the brother was adopted, so no chance the sister could remember it.