Jokes aside, I'm 90% sure she's just kidding, cause that came outta nowhere. She's probably just gonna say something like "see? You don't wanna leave Kaori, so why are you entertaining those thoughts in the first place?".
But any way you look at it, there's no chance of Kayoko leaving Kaori, so there's literally nothing to worry about even if she's serious.
EDIT: also, the author just posted a cute Halloween comic of Kayoko and Kaori on twitter (I **think** it's just an extra, though the text in Japanese might contain spoilers from later chapters, idk):
https://twitter.com/_osakana_man/status/1322955936676261894