Here in Tokyo we have no gravel except in playgrounds. The roads are flat, smooth, and well-maintained. I normally ride a 23mm tire, which I find a good balance between speed and comfort. In my racing days we rode 19mm tubulars in most cases, and 21mm on cobbles. Those were the days when 9 speed drivelines were just coming onto the market, and there were still some pro riders on steel frames. Oddly (or not), the stage records of today are only incrementally faster than those of 25 years ago despite the great improvements in cycling technology and training.
Pros don’t necessarily use what is faster, they often use what is more fashionable, and what their sponsors think will drive sales. No one would say that the baggy clothes worn by MTB and winter sports pros in the 90's were faster than tighter-fitting clothes with less wind resistance. Baggy clothes were significantly slower, but the pros wore them because that was the fashion of the time. Fat tires have been fashionable for the last few years. The current market for "fat bikes" is strong, but no one would argue that 5 inch wide tires are marketed for their superior performance.
Personally, I like the appearance of bigger tires, but I know from experience that bigger doesn’t necessarily mean faster.