It's not really fine with helium either. The lack of oxygen while you're inhaling the helium can knock you out, which could cause other kinds of injuries like head wounds if you fall over. Worst case scenarios of death have occurred as well, made more likely depending on how you're inhaling the helium or if you have some sort of preexisting condition you might not even know about.
It's one of those things that people can get away with maybe 99 out of 100 times, but there is a risk there however unlikely it might seem.