Isekai doesn't equal fantasy. It just means the protagonist has gone to another world. It's not actually a genre. It's like saying male protagonist is a genre.
There are Isekai series where the character isn't going to a fantasy world, but a modern or sci-fi world. Reverse Isekai would be the protagonist returning from another world.
Otherwise, if you say this is Isekai, then anything where any character comes from another world (including demon antagonists and such) are Isekai too. Hell, Mahouka Koukou no Rettousei features a bit about another world. So I guess that counts as Isekai now.
Consistent Definitions for the Win.