One thing that I've noticed is that when Yuu's in the zone he actually shows promise, especially when it comes to his good looks, but that's the problem, he has almost NEVER been allowed to be in it, as he had his self-esteem crushed under the intense pressure of his mother's talent since he was a child, even before the experience with these kids cemented his mentality.
It doesn't help that his mother wasn't exactly a role model parent until she was in her literal deathbed.
Now, about these guys, every single "asshole" character that we've had in this manga was more of a "realist" than an asshole, and weren't exactly "bad guys". The father, the karate guy, and even the sisters herself at the beginning are an example of this, so I kind of expect these guys to be the same, or who knows, maybe they will be full villains.
Regardless, one thing that needs to happen is actual character development for Yuu.