Her ex. is in the story for a very clear reason, besides the "love triangle" TV drama trope - he was the one responsible for causing Ban's confusion regarding love. Harutaka (her ex), instead of recognising her quirks as Yuuki does, just went on and traumatised her, telling that whatever she felt wasn't love. It made her insecure of her own feelings after that. Not that she thought she loved him, but after that she has this idea that nothing that she feels is good enough to be called love.
So him showing up has a narrative purpose as well, which is to clarify to Ban that what she is feeling for Yuuki is indeed love. Since you're following the raws, you can see her reactions, how jealous she is of Yuuki. This last chapter (25) has her seething with jealousy and blaming herself for getting more and more confused as things get more serious.
Also, Yuuki deserved that slap. He was the one who convinced her to continue "training", even though she was clear she didn't know if she loved him. And then he decided to try to force the situation (albeit he thought he was taking the lead there - he was just stupid). He basically misjudged the situation, having taken into account what he felt and what others perceived, instead of actually taking into account what Ban feels.