This time of year might be the best time for an experiment like this since you are only using rollers for form and maybe cardio. Maybe try swimming, jogging, walking, hiking, or some other form of cardio that's not cycling and see if the pain goes away.
I would suggest rowing, but that might be trading one problem for another when it comes to the back.