I think I need to take a closer look at what gears I'm using. These smaller front gears may be great for me.
I have always been fine with a 50/34 compact crankset with an 11-32 cassette and can always find a gear, but do I ever really use the 50/11 combo? Never.