The letter is for Ryouma only.
The proof is in his status, which the novel shows but I don't remember if the manga showed his whole status.
They are of such renown and somewhat reclusive that if you have a title of apprentice for both of them, then most can only guess as you being raised by them.