There is no right or wrong here, this his home so he protects it and they're just doing what adventurers do.
But still, he came up to them and said drop your stuff, of course they would attack. They even mentioned taking him to the guild instead of killing him, he's no different from a monster with a human appearance