Takamura never was happy with the fact he "bullied" Ippo into getting boxing. Yes Ippo was strong, but, in Takamura point of view, knew nothing of the danger. He was a charging bull taking as much damage as giving other. Either he would have died in the ring or suffered so much trauma that he would have been useless for his mother.
Takamura is a brute and a asshole but he is right. Every "champion" so far were monster in their own right. Offering everythings they had to the ring and boxing. Takamura himself know that, without boxing, he would be nothing. Adding the fact that he want to offer all the world champion title he can to his coach and you have a character that see one boxer giving preferential treatmen wasting it.
Yet in this chapter it is clearly show that he is angry. Not a those three, but at Ippo and, probably, himself. I'm pretty sure he's mostly angry at Ippo for having show hope to the coach of "his style is not wrong" and yet wasting it. And I'm pretty sure he know that Ippo is not afflicted with drunk punch.
As it is now, Ippo can't go back to the ring, he would only commit the same mistake as before.