I understand feeling like this. But the whole "not worthy" mindset is such a useless outlook to have. No one benefits. Not the girl who likes you, and definitely not yourself.
I think it's realistic for a teenager like Inuyama to not have the tools to deal with those thoughts, but for myself, as a 30-something "ossan", I fight those thoughts with everything I have. Life's too short, and they do nothing but hinder you from bettering yourself.