It's kinda awkward. To our knowledge they haven't kissed but they've slept together a couple of times (not in a sexual way, and partially because of the pressure by Megumin's mom). While Kazuma seems much more timid, Megumin seems to be much more open about discussing their relationship.
Also, Aqua's completely oblivious to everything that's happening.