There is a guy I follow on Strava that has a Roubaix S Works and he regularly does 50 mile rides with approx 3000 feet of climbing and averages 17 mph. And he's probably in his late 50s and that's on solo rides. So if you are a strong rider a "fast" endurance bike shouldn't slow you down and I'm sure there are people that benefit from them and are faster than they would be on a more aggressive bike.