I don't think the majority of people realize it until they are already beyond that point. I think as long as you can have healthy physical relationships then you are still good.
I think the bigger risk is addiction to porn. Needing/wanting the good feeling from rubbing one out seems healthy IMO. Not being able to bang your wife because she doesn't look like your porn fantasy doesn't seem so healthy.