Jack can see the true form of emotions, it's never black, white, or mono-color. It's like a painting with a variety of colors and shades, Jack has trained to know which color represents which type of emotion.
But Frog gained power through experiments and it only showed negative emotions. Jack also misinforms as he thinks Frog sees the colors the same way he does, Frog also misinforms about this as well. Frog thinks Jack somehow loves humans despite only seeing dirty negative colors.
Jack has always been the one with a deep understanding of human psychology thanks to his eyes. Frog's life has been changed completely because of his eyes. So Jack considers his eye a gift, while Frog considers it a curse.
Frog's life would have been better if he had never been able to see colors.