That's the main caveat, not all fitness is the same and not all fitness is created equal. My peak CTL for the season was pretty high, but I wasn't particularly fast, and got faster in the cyclocross season by decreasing my CTL and increasing my amount of intensity for my race specific fitness required.