Building fitness is not a linear progression. Your body will have peaks and valleys, and sometimes for reasons you can't figure out. Especially as we get older, allowing our body to recover becomes more important, but also takes longer. Recovery might mean being totally off the bike, or it might just mean doing some easier rides for a little while. Also keep in mind that your 10-mile ride to Baldy was probably a bigger workload than your typical 15-20 mile ride. Being that you're still pretty new to riding, your body is still figuring out what the hell you're doing to it.
__________________
Platypus gravelus.