I like a lot of this, but his family is just so weirdly mean to each other so often. Some other characters too—well, it's not horrendous cruelty or anything, just feels bizarrely like some sort of frat-house antics—but it just doesn't quite jive with the whole "let's relax" theme that is otherwise there. I'd hate having a family like his, it would suck.
There's also the questionable hints of "onee-san" fan-servicey stuff, but if I complain about that in this sort of manga I'll never be finished.
Edit: Reading the previous comments and having someone say the author might be into SM... would explain a lot. Like people getting mad at so many petty little things and delivering "punishments" for them that seem slightly unfair and out of proportion. If I play back these scenes and think that the author treats the anticipation of punishment as, um... stimulating... then a bunch of slightly odd moments suddenly make a lot more sense.
I'm not at all convinced the author was intending to write PG-rated S&M here as such, mind, just that if they have such preferences that may have seeped into the work a bit, and that in turn might explain... some things.