Yeah to me it doesn't feel like he has any sort of lingering romantic attachments to Lemon. It's more like he would feel stuck in a really awkward position if Lemon is still in love with him considering he likes her friend. Then add in the body switching, it would just make it even more awkward...