Yup. At 62 with ambitions half my age, I've been forced to confront reality and make some changes.
I got more methodical about training (not counting casual pleasure rides) and had to give myself a rest day after every real workout. So I usually ride about 4 times a week and rest 3 days -- usually 1 day on, one day off, but occasionally I'll ride 2-4 consecutive days, then rest for 2 days. I'll walk and do some light exercises on rest days, but it's mostly actual rest, not "workout other than bike rides."
I made some progress last year with that approach, although I've slacked off since January, between the weather, chronic sinus problems and then the coronavirus inflammation. So I'm mostly in maintenance mode now, not improving but not losing too much fitness. No HIIT, but more zone 2 rides to balance out the tempo pace rides. Works for me, YMMV, horses/courses, etc.