I had this same problem at around age 55 and it persisted for four years. I tried everything from more carbs to more protein and less carbs. Nothing worked. Doctor simply told me that it is part of the aging process. Testing showed I was healthy, not diabetic or anything like that. O2 uptake was normal for my age (not sure if that is good or neutral). Once I accepted what the doc had to say, I forgot about it and then when I hit age 60 it all went back to normal. Totally weird.
2 years later I can ride as normal, not fatigue until mile 60, and am enjoying myself tremendously.
Suggestion, as always, go see a doctor.