Yeah, its also possible he's remembering things differently
His talent became dormant after this traumatic event because we see at times, especially at the start of the series that he's capable of 'acting' even if in weird moments. Its still there so its kinda safe to assume he was bullied off...