Technically, he never affirmed that he was joining any of the factions. He told the first girl that he was more interested in her and that's why he'd join not that he actually was. The second girl he told he that her question was trivial and that her desire is his command. Both seem like agreements to join but are also ambiguous enough to cast doubt.
That maid really did a number on his ability to clearly communicate with women, but he was crystal clear with the trap. A little surprised MC didn't say to their face they weren't into dudes.