@Kshatria Pretty much any armor has some sorts of penalties, or upper limits for stuff like the dex bonus which could lower your AC depending on how much dex you have. I've played a number of characters that were better off stark naked at higher levels than in armor even though they wore stuff like leather or hide armor early on.
The clothes and robes made for monks are often pretty useful in such cases since they might have some nice bonuses, if they don't have super specific restrictions on them either anyway.