For shorter rides a multi-tool and tube can go in a saddle bag. Rain shell and pump can go in a jersey pocket. Not much else has to be carried.
For longer rides or rides in the backcountry I have a small Apidura frame bag almost identical to the Tangle. It does not rub even with thermos, food, shell, arm warmers, water filter, etc. Stuff can't just be shoved in so that it bulges. A little bit of thought should go into how it is packed.
The bag also has to fit the frame. Many size them too big and they fit baggy and all droopy.
-Tim-