I have tried using the radio function on my iPod Shuffle, but it doesn't hold a station for more than a few minutes. I've had the same experience with other mp3 players that have built-in radios.
That said, I have also tried the Radio Shack portable radio mentioned above and several others with similar results. For some reason, here in New York at least, it's very difficult to hold an FM signal on a bicycle.
May I ask why you are averse to an MP3 player? I have one that I use with a small blue tooth speaker on my handlebars and I can't imagine riding (especially all day long) without it. My iPod Shuffle holds several thousand of my all-time favorite songs, so even the horrible moments when I'm struggling up a hill or slogging through my third hour of no scenery at all are like a party.