A narrowed focus on them is fine and even good for a romance-centered narrative. But making their problems/motivations start and end with each other makes for a less compelling story. Even if the world is just a vehicle for their relationship, it should still function as a world they exist in and that affects their lives.
if that is the case, then whatever "worldly event" that transpired would have had to impact Kazuki and no one else, and occurred a little more than a year before the start of the series.
Nothing we've seen from Sayaka or from Kazuki's mother or from any other introduced character suggests they're aware of some external event that has turned Kazuki into the person he's become. His mother's interactions with him strongly suggest she's simply fed up with his moody and lazy behavior, and we're only just now getting anything of substance from Sayaka--but it's all focused around her memories of them as kids, and the texture of their relationship with him "being the strong leader" and her being the weak crybaby who is always helped by him.
There's nothing in their attitude toward Kazuki that would fit with a tragic event occurring that caused him to pull back from the world, at least on the level of the father dying or leaving or something. That would have impacted the mother just as much, and she'd have a point of connection where even if she was tired of Kazuki's behavior, she'd have some semblence of understanding as to why he's acting this way--and instead it all seems foreign to her.
Sayaka would also be privy to such an event happening, what with how close they were all through their childhood, and even with Kazuki's mother (she's allowed to go over and use their bath; not something you do for your child's friend unless you know them well, yourself). But nothing we've been shown from her points to her being aware of the reason for why Kazuki is the way he is now; and something like losing his dad would be on her radar as a cause for the shift in attitude.
That doesn't mean it couldn't be something else, but again--we've gotten nothing in the way of foreshadowing, beyond knowing a timeline prior to the manga's start, and the one flashback of some martial arts contest that showed Sayaka getting first and Kazuki getting third, that coincided with him wanting to throw away all of his placards/awards from that period of his life.
Whatever happened, would have to simultaneously be important enough to Kazuki that he'd throw away everything he'd liked before that point,
and be small enough that his mother and Sayaka either aren't aware of it, or consider it so small a thing that his behavior is unjustified.
If instead it was a crisis born out of Kazuki's loss of identity in being "the leader" for Sayaka, then what we've been shown from the cast to this point would at least be consistent.
And besides -- the world
is still there; we have the sports festival in this very chapter, and the "Hell Relay Event" was brought back by other agents and is what lead to this moment where Kazuki was forced to step out of himself,
and where Sayaka has a repeating of history (last chapter's flashback) and Kazuki finds himself faced with a decision. But the main driving force of the narrative remains Kazuki and his grappling with himself and what it is he wants to do, and who he wants to be--coupled with how Sayaka fits into his life.
