It would be even funnier if instead it's the same dude every time and each women comes into the room saying they are his girlfriend. The harem has become aggressive since he lost his memories.
This is my opinion based on a visit and analysis of Kuga-sensei's Twitter, completed today (10/18/2023 04:06 UTC). Please feel free to disagree and offer new interpretations, but upon re-reading in the order that Kuga-sensei made the posts, it seemed to make sense to me. What do you think?