I sometimes wear ski goggles on really cold (like sub 20F) rides on my fatbike, especially if it's snowing. Ski goggles work better than motocross goggles on super cold days because the lenses are double walled and very fog-resistant. Otherwise, glasses work fine.
One exception really long, cold (20F-35F) gravel rides on days where the air is very dry. The combination of dust, cold and dry air can make my eyes water and eventually completely dry out. This can make it hard to see after ~5 hours, something I learned a few years ago. This is pretty dangerous. I bought a pair of these with clear lenses for $7 shipped for these rides:
https://www.amazon.com/Pyramex-Safet...74630584&psc=1
A bit dorky look but turned out to be extremely effective. I don't use them very often but they're perfect for when I need them.