@Scalim
僕 is "neutral" but also masculine. (Think of it as "how little boys say I".)
私 is absolutely neutral. It's also very boring and very formal. (iirc Dia uses this near exclusively.)
俺 is VERY masculine. To the point of being offensive in many contexts.
A Japanese man would use any and all of these throughout his life. A woman, however, would likely never use any of them but "watashi".
The Gems all use some form of male pronouns. Some like Bortz are super masculine using exclusively "ore". Others, like Yellow and Phos, use the less masculine "boku". None use "ashi" or "atashi", the common "female gendered" pronouns.
And that is just self referential pronouns.
They all exclusively refer to each other with "(o)ni-san"/"aniki" and "otouto": "big brother" and "little brother" exclusively, respectively.
Gender is purely a social construct. This is especially true when the society is monosexed. (In this case, asex.)
And as such, all Gems are male.