somewhat of a guess from knowing other spoilers but, it's probably a bit of jealousy combined with his own growing feelings for her and the guilt when he has a fiancee.
Ah yes, don't forget to tag mangas where childhood friends who grew together and eventually mention something about being like family but one of the parts hold them as a love interest as "incest" too