Couple weeks ago I did have a guy pass me while I was puzzling about something and let my speed drop. I estimated he was about the same fitness level as me, but he was a tad faster on the level parts and I climbed at a faster speed than he did. It was quite difficult to stay well behind him, but likewise I didn't really want to pass. This was a day I was trying to ride easy.
Sometimes I was 200 feet back sometimes 15 feet back. Wound up following for about 8 miles before they turned off. I hope he didn't think I was on his wheel and writing about me on some other bike forum.