I didn’t start cycling until i was your age. I’m 62 now and am faster than I’ve ever been. Not sure if that’s an indictment on my past training, or my current training, but typically, my rides of 25 or so miles average around 20.5 mph right now on all/out effort rides (faster on TT bike). My FTP hasn’t increased much, but my pace has.
Everyone is different, and i see the benefit of “just riding,” but that’s not for me. I love the power meter, HRM, and gps. I’m super competitive with myself (and much younger riders). I have purposeful training involving hard rides, easy rides, and interval rides.
i also supplement my workouts with regular weight training (full body) and eat pretty well.
i am retired, so i have the time to do this.
Not sure about being grumpy after hard rides. I’m usually a little grumpy when i CAN’T ride.