For my riding, it is the cold weather that has a significant impact on my performance.
Realistically, I'm sure that hot weather would be a the limit as you can always dress warmer to mitigate cold weather effects. But there's a limit to how cool you can keep yourself while riding. But I do not ride in extreme heat. In the summer, I get out at dawn to get rides completed before it gets hot. I rarely ride in weather above 80 F. I also live where it is very dry, so sweating is very effective.
I don't know what it is about cold weather that affects me. I have plenty of high-end gear to wear so, I'm able to stay comfortable most of the time. I think it really comes down to breathing in cold air. Though my legs never feel cold, perhaps the cold is affecting my leg muscles.