The line is「会うたびに背が低くなっちゃまうよな」. 背が低い is
"short" (in terms of a person's height) rather than "low" (低い by itself means "low", though it could also be used as "short" in its own right-- 背が低い is explicitly about a person's height, 背 meaning "stature" or "height", as it acts as subject for 低い).
The way he corrects himself, saying 「いや... 違うか / 誰よりもたかいんだから」(approx. "No... I guess that's wrong*/because you're taller than anyone else, right now."), 高い (
which can mean either "high" or "tall") would almost certainly have to be understood as "taller" as a turnaround to the previous "shorter".
So, it was "shorter" and "taller" all along. With him looking to the sky as he says the latter phrase in question, it could have been "shorter" and "higher", since it appears that he's playing with the fact that 高い signifies both "high" and "tall" even by itself (whereas English speakers are more inclined to distinguish between those two ideas).
But, absolutely, he said that she gets (or "happens to get", is the most likely nuance) shorter every time they meet.
*The か in 違うか is used here to indicate doubt in his declaration that what he just said is wrong, or that he supposes what he said isn't correct.