If you think about this from his perspective, trying to make this work would be an absolute nightmare in practice. It's all fine and dandy saying that what others think doesn't matter, but would you actually want to live a life where all your friends and acquaintances think you're a creep? Sounds insanely stressful and depressing.