Bike Insights makes it a lot easier to visualize the difference.
https://bikeinsights.com/compare?geo...b60200252de3eb,
Basically, the Roubaix 54 and 56 have the same wheelbase and BB height. 56 puts the saddle a little bit further back, and the head tube a little bit further forward. I.e. it puts the rider up slightly higher. The head tube angle is a little different to keep the wheelbase and, I assume, keep the handling consistent between sizes. Differences in trail are probably neutered by the required changes in stem size, etc. In other words: Since the bikes share the same design goals, I doubt you'd notice a huge difference in handling between sizes. If you do, that's probably much more your mind than anything about bike geometry.
I.e. you should focus on what fits you better, rather than how it handles.
I'd add that you don't really need a huge sacrifice in responsiveness in order to increase comfort. While endurance bike geometry is a bit more slack, it's not like you're switching to a touring bike; they're still racing bikes. The relevant buzzwords for most endurance bike designs are "laterally stiff and vertically compliant." Much of the comfort difference will be in tube construction, the amount of exposed seat post, and wider tire clearance. The Roubaix also adds its suspension stem ("Future Shock"), which doesn't change the handling.