You can check up what any doctor recommends... It's almost consent to not use lenses especially on the ocean. It is not guaranteed you will have troubles, but you would are rolling a dice every time.
I am not a doctor, but:
Risk of infection
Risk of scratching your retina - if some tiny spec of sand get between your lens and eyeball
And also the salt itself can cause irritation with lenses on
btw, pools are the same on the infection part...