Although it can be a bit of "if fits fill it full" bigger doesn't need to be filled but if they are too small it sucks. What you will find is you need nearly as much stuff for a weekend tour as you need for a month. Camping gear, clothes etc etc. Main difference is food, you tend to carry extra staples on long tours. So likely you'll need bigger rather than smaller bags anyway if you're just running rears..
I have back roller classics, they'd be too small by far on their own. On long trips I have full front and rears, plus a basket on the rack with assorted junk. If I had my time again I'd get Back Roller Pro Classics, for those times I need to carry extra water and food.