Both have their place. I live in an area where the temperature can be 35 degrees f when I begin my ride in the morning and 80 degrees f when I return in the afternoon. Arm warmers provide great flexibility for this kind of riding.
I also ride in the (rather mild) winters here on days when the temperature never gets over 45 degrees. On those days I wear the long sleeve jersey all day.
Long sleeve jerseys are often made of a heavier material than short sleeve jerseys by the same manufacturer, so keep that in mind when making your decision.
Brent