You need to get out more- or ride somewhere else occasionally.
This never riding in the rain bit is causing me a problem. If I did that I would not get out much over here. Only time I have ridden in the dry in the last month was at 20f and snow on the ground (Or ice really as most of the snow had melted.)
But temperature doesn't matter when it rains unless it is above 70f. It will still feel cold. I did a ride years ago with the temp in the high 50's and came very close to hypothermia- and that was with the Goretex on.
So a trip to the LBS is on the cards- Waterproof outer- Goretex is good. Two pairs of leggings- one pair with the thermal underlay. Waterproof socks and gloves and I can recommend SealSkinz
Or just get yourself a lightweight showerproof top that folds up very small. Carry it with you all the time and it will never rain on you again.
__________________
How long was I in the army? Five foot seven.
Spike Milligan