The problem with hell difficult challengers is that the first few floors put so much enfase in dodging, defense and being safe that, otehr than the MC, they are all scared of fighting when they don't know how strong the other is.
It changes as time passes from the parts where he is in the 60th floor but at the beggining most don't even get over the 1st floor because of how scared they are.