Genetics is a big part. My brother in law was fit and one of those wiry strong guys. No problems. Started feeling faint and sought medical help. Ended up having a faulty heart valve and a 6cm aneurysm on his aorta at the ripe old age of 32! Genetics. A pipe now replaces a section of his aorta and he has a new pig valve in there. Gone fro a ticking time bomb to a second (albeit limited compared to his previous lifestyle) lease on life. A workmate had a low level heart attack at 45 due to a blockage. Very slight very fit guy and a smoker to boot. He showed me his scan of the blockage. All it was was a small section of his aorta looked like someone had pinched it. The muscles in the wall had failed. Doc told him it was totally unrelated to his smoking and just something that he sees happen from time to time.
It is widely known now that certain lifestyle habits can greatly increase your chances of medical issues, but sometimes genetics and just plain bad luck can have you questioning if it is all worth it.
On another note, physical activity is just a small piece of the puzzle. One big factor to keep in mind is that (depending on where you look) active people only burn about 20-30% of their daily calorie expenditure through exercise. What that basically means is that any activity is not going to be some magical bullet. You are only increasing your calorie expenditure by a small margin by even being quite active. Most people get into exercise as a weight loss measure and neglect to pay equal attention to their diet, which is the biggest piece of the puzzle.