Everybody asking for a BIG progress in a "Slice of life" manga.
The main protagonist is a socially inept person. Could he even said to Yamada that he felt used or ask her why she did that? (in chp. 42) You just have to keep in mind that he have almost no social skill when you're reading this manga. Then, maybe you can enjoy this.