I actually really liked that. It's stupid and overdramatic, which is characterization. He hears the word baby and gets in his head about a scenario where she's pregnant from a past relationship and then comes to the conclusion that he's going to love her and that baby no matter what. It's the kind of story a teenager would imagine for themselves, making themselves a hero in a made-up obviously false scenario to boost their own self-image.
It's a reminder that he's a dork who's exposure to romance came entirely from games where a dramatic turn like this could happen. That despite his appearance as a nerdy glasses guy, he's not particularly brilliant or cool-headed, he's the way he is around girls cause of a trauma, not maturity. Most importantly, it shows that he really wants this relationship to work and would take even a crazy event like "she's been pregante and I somehow didn't realize it" in stride because he's decided that he's the kind of mature and forward thinking guy who doesn't care about what she did before, just that they love each other now.
It's kind of a mirror to the chapter where Nanjou got it in her head that he went off and had sex all night with the dark haired girlfriend of his H-protag buddy. They're similarly ridiculous scenarios that only make sense to you if you remember that these are teenagers operating from specific worldviews based on limited life experience. They're misunderstandings, but they clear themselves up and let us know more about the people who had them along the way.