Probably nothing. The CAAD12 is a phenomenally good bike and a comparative bargain.
For years I rode only steel bikes. Last year I bought a 1993 Trek 5900 OCLV from a friend. It wasn't completely original -- he only wanted the Dura Ace components, I only wanted the frame, but he built it up with nice Shimano 600 Tricolor/Ultegra parts so it was rideable the day I got it. I really thought the 5 lb lighter bike would be a revolution. It wasn't. Because of the conventional diamond frame geometry it rode very much like my '89 Ironman, but felt a little lighter on climbs.
But we don't have much in the way of climbs here. It's all rollers. Lots of short, steep, punchy sprint-climbettes. On a good day when my legs and lungs felt strong the Trek might gain me 0.5 mph overall on my usual 20-40 mile routes, and maybe a couplafew seconds on shorter segments (I only pay attention to segments at least 1/2 mile long, flat or at least slight inclines). Not a huge difference.
Another friend needed to make room in his garage and gave me a nice 2011 Diamondback Podium 5 frame. He or the previous owner replaced the original FSA crank with Ultegra, so it was a pretty sweet starter. But I got so busy and had a long bout of illness (not COVID-19, just Hashimoto's, a longstanding pesky but usually non-fatal auto immune disorder), so I just didn't feel like building another bike. I was perfectly happy to ride my Ironman and Trek 5900. But the old Trek headset needed to be overhauled so I took it apart and put it away for awhile, only riding the Ironman most of this year.
Last week I finally built up the Diamondback frame. I hoisted the comparably bare Diamondback and Trek 5900 frames and was surprised how comparably light they were. No significant difference. But the Podium has a spiffy tube design and is stiffer, so it should be more efficient on those sprint climbs -- my nemesis.
With a hodgepodge of components -- some Ultegra, some Dura Ace, some 105, and very ordinary old school semi-aero Mavic/Ultegra wheels -- the Diamondback weighs around 18-19 lbs, a bit lighter than the Trek.
I've had two test rides on the Diamondback, of 20-30 miles. Zero difference in my segment times or overall averages. Yeah, the bike *feels* lighter and more responsive on those rollers. But that didn't translate to faster. I'll chalk up some of that to not having the bike fit dialed in yet. I wasn't sure whether I could handle it with the stem slammed but it turned out the spacers were too much and made me *less* aero than my old Ironman. So I replaced the 100mm stem with a 110mm and nearly slammed the stem -- just a very thin spacer above the headset.
I'm going for another test ride now. But I'm not expecting much. It's a nice bike. I'll enjoy it, although the ride goes beyond stiff into harsh. I added thicker bar wrap (Arundel Synth Gecko), fatter tires and latex tubes, and a slightly more springy old school Selle San Marco saddle to soften the ride a bit. But at 62 I don't have the engine anymore to wring out the potential from a "better" bike. Doesn't matter if it's my 24 lb Ironman, 20 lb Trek, 18-19 lb Diamondback. The only significant improvement I can make at this point is to get an aero position I can hold longer. A $32 pair of latex tubes will make more difference to me than a $4,000 frame.
But if you have the engine, and want another bike and can afford it, go for it. Life is short and if the Super Cooties Apocalypse doesn't get us, something else will.