The plague is certainly the largest deciding factor but certainly not the only one. And we actually can't say with certainty that if Ivar character and sword hadn't been an issue that the Lnu would have still ended up attacking the settlement, like this specific raid is literally here because they wanted to obtain Ivar's sword. Also Ivar not playing right into Miskwekepu'j's antics and spoiling the relations with the local Lnu tribe before the plague spread could have drastically changed the response and interactions between the settlment and tribes and thus the consequences of the plague spreading. It's very possible if Ivar hadn't done what he done in the past Thorfinn would have been able to successfully negotiate having until spring for the Norse to withdraw from the settlment.
Thing is, Ivar ultimately cut the hand of the Lnu chief out of concern for Thorfinn, in his eyes, Thorfinn is a utopic weirdo who get bossed around by a woman (Hilde) and he still doesn't know that he's actually a warrior who far surpass him and while he disagree on his methods, he still ended following him on a journey into the unknown because Thorfinn has no concrete proof of Vinland's existence beyond "trust me bro, this old man told me Vinland exist so it must exist".
If he actually knew Thorfinn didn't need help or protection, he wouldn't have cut the hand of the chief.
And while he spoiled the relation between the Nords and the Lnu with the fort they were trying to build being symbolic of that, he still adopted a defensive stance in the end because a fort is ultimately a defensive structure.
It's very possible if Ivar hadn't done what he done in the past Thorfinn would have been able to successfully negotiate having until spring for the Norse to withdraw from the settlment.
We don't know and probably won't ever know, the chief already had a heavy bias against the Nords because of the vision he had and we don't know if he would have the patience to wait until then while a lethal and unknown disease is spreading and killing his people.
While the Lnu certainly hold some responsiblity (a great deal even) for the situation going to shit (and even then none of them had words a tool that is only used for killing other people, and which was the only weapon/tool Thorfinn said would be banned from the Vinland settlement venture), they are the ones who's land is being intruded upon by foreign settlers, in such a situation said latter party holds the greater burden and responsiblity of establishing and maintaining peaceful relations.
How should a leader react if a foreign leader tried to stab him? Remember, the old Lnu chief is the one who attacked first.
Thorfinn wasn't in any real danger but only people close to him knew that, for everyone else it was a clear attempt to assassinate or at least harm their leader.