I'm just a layperson too - but one that has used (soft) lenses a lot while swimming, and I've had no problems with infections in pools or lakes, not managed to get sand beneath my lens and didn't experience particular irritation even when swimming in the mediterranean (which to my understanding is even saltier than your average ocean).
I can't speak for "hard" lenses though - nor would I suggest waiting until you are on the sandy beach to actually insert your lenses. Do that a bit earlier if possible.