I think about this topic a lot. Carbon Fiber vs Aluminum frames.
My bike is a 5 year old aluminum CAADX with 105 components. I bought a set of aluminum tubular race wheels with challenge tires, which made a huge difference. I hated the stock cantilever brakes and upgraded them to mini-vs which work well in most conditions, but are not as good as hydro discs. My bike has served me well, but I've been thinking of upgrading for a few years and carbon fiber/disc brakes seem to be the obvious choice. The bike I see the most at my local series is the SuperX, so I went and test rode one at my LBS and wasn't as blown away as I thought I would be. Hydro disc brakes are an obvious huge upgrade (and it's pretty much impossible to find rim bikes in this category now anyways), but for $1500 or so difference, I don't think the carbon fiber frame made as big of an impact as I was expecting. I mean, the bike definitely felt a little lighter (due to both the frame, and a higher component spec) which isn't insignificant, but the overall ride and handling didn't seem dramatically different. I walked away thinking maybe I should just spend the difference on nicer wheels. In the end, I didn't do that either and am still just riding my old bike.
This could partially be a result of the test bike having stock wheels with clincher tires, which are definitely holding it back. And since my current bike has rim brakes, any new bike I buy would need a new set of tubular wheels regardless. Still, I can get an Ultegra equipped CAADX with hydro disc brakes for around $2k. The equivalent SuperX (Force 1) is $1500 more.
I dunno. I'm sure in a blind test with a stopwatch the SuperX is faster around a CX course, but all other things being equal, the difference in frames from a seat-of-the-pants perspective didn't feel like that big of an upgrade. Maybe I need to look at other brands or try out a friends bike at a race sometime.