They neglect it in the manga but it's explained in the web novel at least that those masks all have perception blocking capabilities. It's a mental block not a physical block so others can't comprehend the face they're looking at beyond basic features like the wearer has eyes, a nose, a mouth, etc..