It has to be tucked down. Left or right it'll slip out.
Some advanced crossdressers will take the extra step to maneuver the balls back into the body cavity, tuck it in, then use medical tape to tuck over the shaft.
I don't remember how I came across this forbidden knowledge. Might've been 4chan.