This is exactly how misunderstandings happen. He's trying to play cool, but the only thing he'll do is ruin his relationship with Mildred, by either swapping paintings or acting like he only wants the mansion. And he'll have no one to blame but himself.