MC has fought beasts and monsters since early age, risking life and limb, plus saving damsels on the side, to get strong enough to become a hero. What's a random third son of a minor noble in comparison to what he has already gone through? Even if he was momentarily pissed off when called trash, in the big picture such matters are too insignificant. Contrary to what edgelords and populist politicians would have you believe, you aren't happier and enjoying your life by being perpetually angry. In fact you will only grow bitter and able to enjoy less and less things in life. The MC is wise by trying to patch things up with the fool. They are stuck together anyway for this trip, so it's better if they can treat each other cordially.