The first time I rode a Specialized Roubaix it was an SL3. I remember being scared out of my wits on a very fast descent as it handled like a truck to me. Jumped on a Tarmac right after that and felt at ease. Body position and weight distribution make a difference. You should notice a difference not only on the climbs, but also in sprinting and cornering.
Now, if you are satisfied with the body position and handling of the current ride, just get a top of the line set of wheels to cut a few oz of rotating mass and save some money.