I am 57 and retired. I generally try and do 30 miles a day 5 or 6 days a week BUT I am never in any hurry. If I see something interesting I'll stop and admire it. When I see a comfy spot to stop and take a break, I do it..
For me cycling is not only about fitness but equally about enjoying the ride. Enjoying the ride includes stopping and taking breaks to enjoy the surroundings whenever I want..
I see nothing wrong with you going 5 miles and stoping to relax and enjoy your surrounding. Clearing the mind and relaxing is a part of life that should be done often. The fact you cycled there is better than driving you car there to do the same..
Relax, enjoy cycling and soak in all the surroundings that come with it..
My .02