No cat 5 races are not very fast. People are just blowing smoke up your a**. If there's a genetic freak, they'll roll off the front and the pack will be happy to let them win so that they will no longer bother them with an uncomfortable pace.
Of course, it's all relative -- it's slower than a cat3 race, which is slower than an elite race, which is slower than many European races. That being said, I'd be willing to bet that some of the riders who didn't get dropped couldn't average 21.5 on a training ride. You got dropped because of your cornering skills and your ability to draft efficiently. Like most people who ride solo (or your average MTB'er and triathlete) you have to learn that there is pretty much NO PACING in a criterium. You have to be willing to sprint and put yourself in a hole to catch and keep a draft. Done properly, you're sprinting maybe 10% of the time, riding fairly hard 50% of the time, riding easy 25 % of the time, and probably not pedaling at all 15% of the time.