I have a non-padded pair of Bontrager tights that are awesome. Thick enough to provide some wind protection but not too warm. I wear them over regular padded shorts. When it's "really cold" (by my standards - roughly 35F) I'll put long underwear over the cycling shorts and then the riding tights over that.
The lowest temp I've ridden in is about 20F. Usually it's snow or ice that keeps me home - not the cold.