Has anyone considered that Unoki may have bullying induced depression?
He's been the lackey for so long that he says that it's his natural standing in the world. He goes out of his way to demean himself through chores or as the recipient of physical violence, not because he's like Matsuri, "Who gets happy over anything" but because he thinks should be useful, and doesn't feel like he deserves to be happy.
This chapter, he suggested past that he might not seem useful, but that Toramaru was better off without him. He's already said that he wasn't worth being a love interest, and feared not being a worthy lackey, but now he's saying that he is definitely not worthy of being a lackey. "No, I'm not trying to excap-" = "I really feel like I've overstepped my bounds just thinking about you in a romantic sense, and am disqualified as a lackey."
Not as a friend or as a living person, but as lackey: someone's thing to be used. Unoki has not referenced himself as a human, every single time it's as a object that belongs to someone else.