I mean, I can definitely imagine it in such a way that it does sound quite natural, it's just that it doesn't flow as well to me as something like "I'll be looking forward to it!" with "You better be!" as the response. [Which wouldn't be accurate]
I was worried that, as you said, some people...