I've tried all the above. I do tend to ride faster than average and didn't like futzing around with panniers - especially since I tend to ride home into a headwind every day in the spring and don't need a wind block.
Messanger bag (with cross strap) worked best for me. Gave me the flexibility of a backpack, without any of the sweat (and lower center of gravity). Timbuk2 has a variety of sizes.