Hoo, boy. He actually said it to them.
There are definitely going to be people who will argue, and part of me wonders if memory magic is going to play a role in the future. It's an unusual situation, since despite what 'cooperation' they may have, Baal clearly still needs access to Iruma (or at least the ring) in order for things to progress - him just going isn't in the cards.
Of course, that also means that's another side who would want to interact with him before he leaves, and that's a very easy thing to make happen...