隣 can just mean "next" (e.g. "next to me"), so instead of "him, my neighbor" it should probably be something like "him next to me" or similar
this is also the ambiguity he reacts to with the term, as unlike "him, my neighbor", it can naturally be turned also into "my boyfriend next to me"
I guess it's kinda funnier this way though