I think the story is not yet close to an ending but someone mentioned the LN 4 could be the ending so who knows. Yoshida is implied to start to develop more feelings for Sayu unconsciously, one of the guys Sayu lived before arrives and that's the next arc with basically blackmail for sex story but resolved without many issues.
Later Yoshida remembers his first love from highschool, a senpai who i think was his girlfriend and first (and only i guess) sexual experience too and basically they split up because she graduated first and no one tried to remain in contact but he thinks he has never loved anyone like her. After some small extra chapters his exgirlfriend from highschool is transferred to his job as a new boss (kind of like Gotou) he overreacts when he sees her and everyone is suspicious, when he arrives that day to home Sayu sees he's acting weird (basically thinking about her) after asking him what's going on he tells her about his ex and she gets worried and jealous.
Some days later Mishima is also jealous and talks with Gotou to try to do something about the new rival. Meanwhile the ex of Yoshida talks with him a bit and after some days pass she ask him to go to out with her to talk more about them, after eating and drinking she ask him if he wants to go to a hotel with her, he refuses saying she does not like him and he wants to do it only with a girl he's going out with and she asks if he has someone he loves, he says Gotou but after learning they are not dating she insists, he wants to go but he still refuses because he thinks Sayu is waiting for him and not arriving all the night is too much without talking with her in advance, anyway the exgirlfriend remarks he has changed a bit in some things but is still the same in others, (the story implies Yoshida just looks and acts like a nice guy taking in consideration the best interest of the others before him from the point of view of the people who don't know him well but his whiteknight stance is also an excuse for his own convenience and fears of acting about something, not taking into account the feelings of others) flirts with him a bit more and they go to home but she asking him to go out again someother time.
