In changing temps, Merino wool is your friend. (My favorite brand is Smartwool.) It wicks moisture away from your skin and keeps you warm even when damp.
Cotton is not so good. It traps moisture, so it is only comfortable in moderate temps--hot weather or cold weather = both bad.
A beanie cap under your helmet can make a big difference. If your head stays warm, it's easier for the rest of your body to stay warm. The cap can be easily taken off or put on as needed and kept in a pocket when not worn.