@nobodygattaknow: it's a self-insert manga, so men are not required to actually have facial expressions
Also, there's a possibility that this is a nuanced translation as 彼女 (Kanojo) can be read as "her" and "girlfriend"
So MC kun may just be saying that "dont bother her" or similar