Tatsuya is so charismatic that people believe he is the one in charge no matter the situation, to the point that Kasumi saw him as someone to blame.
But his thinking was logical in that it's the Mitsuya family who needs to move. I guess this is another proof that he is lacking the emotions that teenagers typically have.