The combat primarily serves as a connection point for events. Ryuu is the swordmaster candidate, she's supposed to be strong. The combat has never been the focus. The panels are easy as hell to follow, I've seen much, much worse. I'm not sure where you even got that impression. The story certainly doesn't go in a random direction, there is reasoning given every single time there is a change in direction, which is almost always due to the main plot. To say that the overarching story is neglected is just plain wrong. Every single significant event is connected to the overarching story. The demon king is exerting his influence on the world and it's Ryuu's job to stop him, but her brother is trying to steal her right to swordmaster from her. What the hell HASN'T been connected to that?