Even on the flats, there are numerous minor slow-downs, requiring some degree of acceleration to get back up to speed. There might be hundreds of these during a typical 50 mile ride, and they certainly add up. But subjectively, I've ridden 16lb bikes on flat rides, and not noticed any advantage to losing 4-5 lbs, compared to a typical 21 lb road bike. If the OP had a 20 kg bike on the flats, he might notice a difference, but @10.9 kilos I don't think a few pound less would be of any help.
Pros often have a bike they use on regular stages, and a much lighter "climbing bike" for use in the mountains. Why don't they just ride the light bike all the time? I'm sure there must be some very good reasons behind this, but I could only guess at what they are (durability? different geometry for a climbing bike?).