If I recall correctly, English is the language used at Chaldea. The staff—excluding the Servants—come from all over the world, and Chaldea itself is not located within the territory of any specific nation.
Upon being summoned, Servants are endowed with knowledge of all common human languages (though if an individual Heroic Spirit was illiterate in their original language, they would still need to learn how to write).
Masters, however, require an automatic translation system whenever they visit regions where neither English nor Japanese is spoken.