Well, often "fluff" is the word used as the opposite to "crunch", where crunch is everything related to raw numbers, lines on a map or RPG stats and such, while fluff is the less quantifiable stuff, like interpersonal matters, character growth, world building flavour and philosophical changes et al. Just expanding on this in the case that the OG comment you responded to meant this different kind of fluff.
But I definitely get what you mean and agree. If the "fluff" in either of these two meanings didn't really matter, then all we'd really want to read would be recaps and excel sheets.