This wording makes it sound like he was expecting her to NOT be wearing underwear, when in reality he was expecting her to be wearing more than just underwear.
Easy fix. Just shove the word "only" in front of "wearing"
Fran...
so you were
only wearing
underwear
underneath
the robe...