Pfff, he is totally at fault here. Just the fact that she entertained the thought that she is holding him back could have been avoided if he were to be less awkward around her.
Remember, it's not only actions that may cause problems. In-actions can just as well cause problems. And after a point maintaining the status quo is just not possible.