In terms of speed for a given power, assuming two bikes are similarly geared and aren't more than 2-3kg different in weight, and on basically flat ground, I've found that the biggest bang for buck improvment can be gotten by going to faster tires. I get about 2-3km/h faster on my 9kg Ti road bike than I do on my 11.5kg gravel/commuter bike, but the primary reason for this difference is the tires - the road bike is on 28C Vittoria Corsa Controls, while the gravel/commuter bike is on 36C Michelin Power Adventures, which are decently fast for gravel tires but not at all as fast as a dedicated road tire. When I had dedicated road tires on my gravel/commuter bike, it was significantly closer in speed to my road bike.