That's fair. I'm mostly speaking from a meta-sense of the story, in terms of how the tone of what we've been shown to this point is rather "light" when it comes to the main conceit of narrative tension--that is, Kazuki going from being this outgoing, bright child to a sullen & moody teenager, with whatever causing it being left unsaid so as to generate interest and intrigue for the reader.
Everything that we've seen from him, from his mother, and from Sayaka suggests that the "stakes" of the story are rather tame, compared to something like an absent/dead father, which would be a real shift in the style of storytelling. Especially as there should have been some hint by this point--even if not stated but merely shown in the background, or in Kazuki's way of carrying himself specifically--to point to this series pushing toward a "heavy" topic like that. Parents dying when their children are young, or one member abandoning the family, are elements in storytelling that push the writing in certain directions: they are by definition more mature and serious, because the real-life parallels are mature and serious for the people experiencing them.
It's a compelling narrative device, but I just don't see it working here, because everything's been presented as very lighthearted up to this point. Kazuki's just a moody teenager who's running away from himself; his mom's fed up with his nonsense and she vallicates between berating and browbeating him, and gushing over Sayaka as she compares the two; Sayaka's presented as a ditzy airhead who's also become "perfect", and against whom Kazuki is constantly measured against (and found wanting).
If we suddenly get "oh by the way a year ago Kazuki's dad died/left and that's why he quit martial arts and stopped applying himself in school or leaving his room", then it makes his mother's treatment of him rather strange; she's treating him like a lazy adolescent, not like a child who recently lost his father and has since lost his way. And Sayaka is close to Kazuki and his mother; she would also know if something like that happened, and seems straightforward enough that there being zero mention from her by this time would feel like a plot hole.
I think if the author wanted to take that serious route with the story's plot, it could work--but no groundwork has been laid to indicate to the reader that the story itself is trending in that sort of serious tonal direction, and none of the characters are behaving like specter of a missing/dead parent is hanging over them. So dropping it in now, would make it feel like a different series entirely.
Whereas if Kazuki simply found himself bested by the childhood friend he was always effortlessly leading around, that sort of blow to his ego and his identity would just as easily & effectively create the current state of affairs of the series. It fits the tone of the story we've been given, and the stakes of "boy loses his sense of identity in adolescence and has to grow up and find himself" is a great premise all on its own, and doesn't need to rely on something as drastic as dead/deadbeat dads to be compelling.