Arm/leg warmers and a thin shell are the easiest things to start with and take off. They can all fit in jersey pockets.
If things get really sweaty it is always possible to remove underwear layers and stuff in pockets. I have wanted to do that several times but it was never bad enough to warrant the hassle.
Make sure to start out a bit chilly on such days. I dislike doing that but have slowly trained into it after suffering through a few sweat-fests in mid-winter.