Back then Shunin was dating Akira and was in love with his sister in law, I really doubt he would have an affair with a random woman in his office, especially considering how much of a bitter person he used to be back then.
Honestly this manga isn't as good as it's predecessor. Still nice enough though.