He could be trying to save her from something back then, and seeing how the "brothers" treated her, I can imagine plenty of possibilities. As I understand, Takako said in this chapter that it wasn't a romantic affair.
Ok, so I had the same question when I was translating it, but in page 35 when Takako is talking about things she got out of her relationship with Narihira she says that "she got to hear about Michizane" and "she could see the cherry blossoms".
It struck me as odd that she would mention the latter without prior reference to it. Afterall, before saying that she got to learn about Michizane from Narihira she was already refering to Hakubai as Tamamushihime.
I must confess that in the Japanese text she says 山桜 (yamasakura), which is the wild variant of the tree, but I didn't have space to make that distinction.
So my theory is that she wanted to see the cherry blossoms, but they wouldn't let her out of the house, so Narihira decided to take her to see them in the middle of the night, without anyone finding out. Unfortunately, they were discovered and they had to escape from their pursuers. And the rest... happened as shown.
But this is just what I think. I could be 100% be wrong. Also, because it's very difficult to see anything during the night. It's pretty dark out there.