I wouldn't say they were afraid to confront their feelings, but this part is open to interpretations (which I like about it).
From how I see it Himmel held back on his own feelings because he understood that his life is fleeting compared to that of an Elf, and properly conveying his love to Frieren may as well curse her with more grief when his time is up, and for someone who can live up to millennia that might as well be hell.
On the other hand, Frieren was ready for it. But she also understands that the kind hero Himmel that she knows wouldn't do such a thing. Which also implies that Frieren had hoped Himmel wasn't as kind a person she take him to be.
So in a way, it wasn't fear. It was an unspoken mutual understanding. Which breaks my heart more.