My disdain for racer boys seems to have kicked in right around when I got a lot slower.
Anyhow, I don't think it's silly to participate in a fascinating subculture. I have a couple of Eddie Merckx (and Coppi and Anquetil) replica jerseys. I don't pretend to be them (I did when I was a kid); I simply like feeling connected to the history of cycling some times when I go for a ride. Cycling has a lot more to it than competition (I believe this is
Hypno Toad 's point, correct me if I misspeak for you) and if I recognize that I am not competitive with more-fit individuals, it doesn't make me less-legit unless we're having a race.
Sometimes it's boggling how people cannot imagine a purpose for riding beyond racing.
Feels good, man.