Are you trying to stop from the hoods position? Vintage sidepull brakes were not designed for that. They are meant to be braked from the drops. Move to the drops, then brake. People did brake from the hoods, but really only to feather their speed for the most part.
If you prefer to brake in the modern style, yes, replace the calipers with dual pivots, and replace the levers as well.