Punishable by death, and the deaths of the slaves who learned if I recall correctly.
A rather neat trick when shortly after the Civil War during Jim Crow they instituted the Combo of the Literacy Tests for voters (Because the expected number of literate African Americans was close to Zero)...
The anime is unfortunately just bad - the focus on swordsmanship is fine and all but at the cost of a lot of context it makes the whole story even more unrealistic than bland than the source material.