I am in Tallahasse these days so summer is the tougher season to ride during the relentless heat. That said since I ride in the early AM I do ride in sub freezing temps some of the time. It isn't a big deal. I have been enjoying the trails year round and not riding the road bike much. I have some nice single and double track trails that start closeenough to the house that I can ride right from the house with only a short hop of pavement. The trails connect into several other trail systems so I have lots of nice options.
I might do more rowing workouts on my Concept 2 Erg during the relentless summer heat, but still ride and I think that the erg helps rather than hurts my overall fitness anyway.
When I did live where there were winters with snow and ice I tended to trail ride on the MTB and get on snowshoes or XC skis if the snow was too deep. The most recent years when I was up north I was more into trail running than riding so the trail riding wan't daily anyway.