Wind per se doesn't bother me. A headwind is a chance to work on form, and since you're going slower you see more of the scenery. It's a matter of adjusting one's attitude. I ride for a specified period of time, speed/distance are just the output of the process, not goals. Therefore, a low average speed doesn't bother me, and having to take extra time to complete a specific distance is not applicable.
However, as others have said above, I'll avoid routes with heavier traffic and narrow shoulders so I don't get blown into traffic by cross winds (gusts over 35 mph, maybe?). If possible, I'll choose routes with fewer trees (if I'm visiting family in RI, this is usually not a problem), but if that isn't possible I'll skip the ride. Anyone who has a yard with trees has found even smallish branches driven several inches into the ground by falling and/or wind. Not something I'd risk for a ride.
Weather related reasons for canceling a ride (in order of likelihood that I'll cancel):
I don't ride in the fog.
Thunderstorms - no ride. Regular rain - maybe, depends on temp, overall motivation to ride, etc. The coldest I've ever been in my life was on a ride in the upper 30s °F with rain. I shivered violently for more than half an hour after that one. This from someone who grew up in Wisconsin and delivered papers (and, lets admit it, went skating/sledding/etc.) in temps of -30°F and lower.
Snowy rides - no, I'm strictly road, though I would like to get some studded tires and try it.
Ice on the roads - this is a judgement call, depends on whether it's patches or significant coverage (e.g. roads are uniformly wet/icy).
Wind - as above, if the wind gusts are in the range of 35+ mph and there is not a good low traffic/wide shoulder/treeless option, no go.
Cold - I've ridden in temps down into the single digits (°F), cold alone is usually not enough to stop a ride. I ski/hike/backpack, so I have plenty of cold weather gear, and I know how to use it. It can be de-motivating, I admit, so I have occasionally decided not to ride in the cold.
Heat - never cancelled a ride due to heat, I just sweat more and probably ride more slowly. Usually my feet suffer the most - they are closer to the hot pavement, on a couple of rides I've noticed hot feet. That said, while I've ridden in weather with heat indices > 100°F, I'm in the NE, so I've never experienced desert heat or Australia outback type heat. And, I'm getting older, so this may change.