I am fairly certain that a Roubaix rides fairly similarly to my Tricross, in terms of steering quickness etc, so if on reasonable roads, even 25s can be alright for comfort--but yes of course, on some really bad roads, my 28s can seem a bit harsh (and as nun says, on soft stuff and in the wet, 25 or 28s arent ideal)
I ride 25mm tire most of the time and when I get on 28mm now they seem enormous. I'd probably go with 28s if my Cervelo had more room for them, but much bigger than that and I'm sure I can feel the inertia of the heavier tire and I seldom go on ground that needs a really wide tire and the rolling resistance benefits above 28mm are diminishing. The last puncture I had on my 25mm ultra Gatorskins was 2 years ago.