I'm mainly playing devil's advocate, because I had someone make that argument to me recently, a sedentary 65 y/o woman with bad knees. And while I don't totally buy into it, there may be some truth to it. Maybe if I were to refine her argument a bit, I would phrase it, "Doesn't a person need to be in reasonably good health to begin with, in order to be able to get optimal benefits from an exercise program, and aren't the benefits obtained directly proportional to how healthy the subject was before beginning the program?"
That would be difficult to disagree with. And even though it's still no excuse for not exercising, I think it's at least logically-sound.
My own view is that you take the first step, whatever condition you are in, ride a bike 20 feet if that's all you can do. And you keep building on that base, all the while improving the state of your health. Improved health allows greater and greater exertions, which boost health, allowing greater efforts, and so on. It's a synergistic process, so to say "only people who were already healthy exercise" really misses the point somewhat.