Well... if the enemy only has arrows, the only thing you need is a cover over the rowers heads.
I don't know jack about carpentry or ship building, but that probably shouldn't be to ohard, there's lots of designs to do that.
e.g.1. most people can take cover within the hull of the ship, underneath the main deck.