The dude actually made a pretty decent life for himself, coming from such a humble beginning. His friends' fate demonstrate what happens more commonly.
If lily had been there she might have had gotten hunted for being a succubus, also even if she wasn't one being the cutie she is she might have gotten kidnapped by some loli kidnapping slavers or gotten the fugly interest of some noble fatass so lily not being there is the best choice.
Yea, if Lily had been there, she would have quickly found herself in a brothel. This look at Hajime's past was a lot of fun. Maybe we will get to see where Lily came from next - not slums by her cloths.