I almost always wear a base layer. I tend to "run cool" and rarely feel like I'm hot or overheating on the bike. I mostly like the additional compression and feel of a base layer, plus it keeps jersey seams and bib straps from chafing on my skin.
When I ride without one now in hot weather, the jersey feels like it's sticking to my skin more.
I do all my riding in the early morning, so I rarely ride in temps much above 80F. I suppose if I were riding in mid-day summer sun with temps in the 80F+ range, I'd likely ditch it.