I broke my hip in a crash 5 years ago and got a "post and beam" repair. My doc wanted me off it for longer than a hip replacement so I didn't get back on the bike for 5 months. That was into winter so I didn't really mind. Once I got back into riding I didn't notice any limitation from the hip while riding. No pain, no tightness. I do notice a slight decrease in the ability to swing that leg over the saddle so I have dropped cowboy starts and most cowboy stops.
Unrelated to the hip, age related drop offs are more of an issue. You have time to spare but watch out in your mid 60s. I am approaching 72 this summer and I have noticed a general age related drop off in my ride distances and speeds over the last two or three years. Friends who are the same age report the same thing. From what I read around here, if you want to avoid that, you need to keep up a pretty active training regiment. I only ride for fun so I don't care if I get passed.