People are coming down hard on Tarou for all this self-jealousy but what you may not know is this is actually pretty common in amnesia cases.
Your sense of self is tied up in your memories. I have my own issue with this series going down the amnesia trail in the first place, but as far as Tarou being nonsensical right now, this sort of behavior is exceedingly true to life. When all people focus on is getting your memories back, a strange disconnect can happen where it's like everyone who is supposed to love "you" sees you as an impostor holding up the return of their loved one. I don't know how many amnesiacs fall in love with their previous love one to the point they're jealous of that exact situation Tarou is going through, but I can't really discount his feelings at all.
Again, kinda hoping the amnesia angle ends soon since it's about the most cliche drama route you can take these days, but as far as a character goes, Tarou is going through a pretty traumatic situation right now.