I had a similar problem, which got progressively worse until I could barely climb three floors without sucking wind. There were no symptoms whatsoever if I didn't exert myself, and I had no issues riding on level roads. Finally saw a doctor: a CT scan showed clots in my lungs, aka pulmonary embolisms. Basically, I was a walking time bomb - one more clot could have dropped me - and very lucky to have caught the problem.
I'd see a doctor about it, especially if you didn't have the problem when you first started riding. (I'm 66, was 58 at the time, pretty much where you are.)