Cake is definitely the safest choice for a gift considering just how much Kaoruko likes their shop's cakes. She'd be thrilled even, if she were to learn Rintaro baked it himself. Cake also allows him to bond with his patissier dad, which makes it even better.
While I'd prefer to see him give her a non-consumable gift so she can keep it long after her birthday is over, I'm sure that requires him to know her a little better; where at this point he does not know her that well (yet). A hair accessory would be weird if she doesn't wear hair accessories, for example. In her case a new hairband might work, but ideally he needs to know something like her preferred color or style.
Anyway, cake is great.