I think the more we get of a character analysis and the more we explore of the psychological state of the main character, I think it'll be easier to empathize with him or at least make it so we view him as less toxic.
To be fair, we don't know what he was talking about with his Ex that start the argument, and she did throw the first punch, and he DIDN'T hit her. He pushed her to the wall to make her STOP hitting him, which is clearly self-defense as a means of diffusing a situation. Granted, his inability to see might have made it so he didn't know exactly where he was pushing her, but I think the way he presents himself to her makes it clear he regrets his actions.
So far, his ex is the only person he isn't rude towards, and I think that speaks a lot to how he still respects her and wants what's best for her. He just knows he isn't the person for that.
Part of his character I really like is that I think he's putting up a front of an inapproachable, unsympathetic bastard. He doesn't want people to get involved with him because he thinks that'll risk them getting hurt inadvertently, and so he does everything to push them away. He, himself, definitely hates the way he is and so judges himself very harshly, and he ends up taking those insecurities out on others. That's what I think the scene in his room at night is symbolic of-his crowded and confused mental state that causes him to lash out without knowing what he's striking at, which only prolongs his own self-loathing and destroys his own image.
I do think you can tell stories about mentally and emotionally traumatized people and make them empathetic, even if they're doing bad things. One of my favorite films is Pink Flyod's The Wall which is about a man who isolates himself from society and involves Pink doing a lot of questionable things. (He invites a groupie after he finds out his wife is cheating on him, intending to cheat on her, but ends up snapping and she runs away from him, for instance.) I hope Sensei here ends up with the mental health he needs, and that we get more insight into why he acts this way, because I think there's potential for a lot of meaningful development.
Though, I'll readily admit I have a habit of playing devil's advocate for a lot of morally questionable and dubious characters.