I suppose it depends on the purpose and need for the ride. If commuting, I'll typically just leave earlier and cope with the wind, whatever the speeds. If above 20-25mph or so, though, I'll be seriously grumbling.
If wanting to do a pleasure ride, if it's above 20mph I'll likely do something else that day. I'm all for incorporating wind into some training, and that's fine. But, for pleasurable rides there's just something frustrating about wind.