Consume the media of that language and learn to talk like the people you listen to. This is the advice that all ESL teachers will tell their students, and it applies to all languages. Use what you learn or forget. Learning is one thing, but practicing is another. If you can converse with native speakers of that language, or people who can speak like native speakers, then that will help. Otherwise, do your best to remember what you've studied.
You don't necessarily learn Japanese by watching anime. You are improving your listening comprehension. You don't need to speak like Quagmire if you watch Family Guy, but you must understand what they are saying in English. Wanna learn Korean? Try to understand the K-pop songs you listen to.
Those smartass who call you names? Bet they can't even speak their only language properly. Ignore their noise.