Fitness is multidimensional. It also depends on what you're training for. I know that when I average 18 mph on a certain hilly circuit, I am fit and when I get close to 20 mph, I am very fit. I tend to measure fitness looking whether I cross my Power Duration Curve to get personal bests OR if I climb known hills faster (less time). If I am doing that consistently, I know my fitness is better. I do not trust my Garmin at all. In the past 6 weeks, my power levels and personal bests have been getting better and better, but my Garmin says my training is "Unproductive" and has lowered its SWAG of my VO2 max from 73 to 61.....what a joke. Hills on the other hand, do not lie.
There is speed, aerobic capacity, lactate threshold, and endurance.
Climbing is a special problem because the numerator and denominator tend to go the wrong way as we age. My demoninator went bad in 2020 covid and am working to fix it.
Where I live, the longest climb is about 8 minutes and the best steep climb is 5 minutes. These stress different pathways than a 30-60 minute TT or a long climb. Repeats on my short climbs are perfect for criterium training but not so great for ultra endurance riding. YMMV