It's not like it's too late for Asahi to go to college and pursue something else that's closer to what she wants out of life (or would want, if she allowed herself). That's why Subaru asks: "Who's going to college? You or me?"
I doubt Subaru is serious about not going to college - she's asserting her independence and trying to get Asahi to do the same, so they can be fellow sisters instead of this quasi-parental codependance thing they've got going on right now. It's quite possible she's already picked a less expensive college that she can pay her own way through - or she's decided to enter a trade instead.