@weyr I don't understand your logic. I've seen several mangakas have their female characters claim "sexual harassment" because of an honest or offhanded compliment so they can hide their embarrassment. It wasn't meant to be taken literally since the mangakas usually show the females liking or even wanting recognition.
Your intent matters as much their impression. Your doctor isn't sexually harassing you if they inform you that your arm is fine, or your leg looks good during a checkup. In the same vein, your intent shouldn't be disregarded if you, by accident, negatively affect someone.
I've read the argument only the impact matters, not the intent. That's hogwash. If you have good intentions, you show them through your actions, not empty words. So, if you act well-meaning and genuinely compliment a woman's hair, you shouldn't face being labeled wholesale as a sexual harasser if she doesn't like it. You should be able to apologize and discuss the boundaries of what can be said freely as to not discomfort them. People make mistakes. Why not allow people to correct their mistakes?
In short, don't make assumptions, and if you have a problem with something, discuss it and then act. You won't incorrectly label someone or their actions and no one will hurt the innocent.