Well, if you look closely, Nagi's "brother" has ahoge hair (that weirdly-pointy-haired that often found in every idiot MCs), which Nagi doesn't, so I'm pretty sure Erika's brother (and Nagi's by blood) is real and the father somehow tries to hide him, or like the other theory says, that brother has engaged with... You know who.
Remember, we're talking about the author of Yamada and The Seven Witches, that twist-after-twist Manga roughly ended in 200+ chapters, so it's too early to tell which one's lying and which isn't.
Nagi haven't talked with his biological mother, right? Neither she ever seen again after the first chapter, only that photo is the clue of her (which I don't buy it if it's her in the photo), so I'm guessing she has the key to connects every dots.
You're too naive to think that Nagi's harem will only between Hiro, Erika, and Sachi. We haven't seen The Senpai, The Kohai, The Yandere, The Kuudere, The Bitch, The Gloomy, and every type of girls that always come in MC Harem's life. I bet Yoshikawa-sensei has ton of tricks up her sleeve for this Manga, so trust her and her works, weren't we satisfied with the ending of Yamada, right?