Ha ha, it took me a while, but as the earlier commenters said, the whole joke is that his "nervous glaring" and otherwise innocent comments to his girlfriend comes off as "the gloating stare of an asshole" and "arrogant bragging" to any observer (like the reader.)