We may be talking about two different things - spirited vs race pace.
I've done group rides with both a 48x15 fixed gear and a Niner gravel bike but I wouldn't bring either to a race pace road ride with the big boys. I'm talking about 40 or 50 miles at zone 4/5 heart rate where your legs are shaking after the ride.
I've lost a bunch of weight and have gotten much faster this winter thanks to an actual structured training plan. As I ride with faster groups I find that my bike is more of a limiter, that's all I'm saying.
B group on a gravel bike all day long, no problem, but when the junior racing team shows up and fast guys are on Argon 18's with Bora wheels and Campy Record EPS or Dura Ace Cervelo R5's, then a 21 lb Niner isn't going to cut it. Cipollini or Sagan might be able to keep up but I need every advantage I can get and there are rides where I would have happily paid for a few more percent.
-Tim-