Our teenaged sons gave my wife and I so much greif over our riding clothes before we ever left the house in them for the first time that anything the general public said or did while we were on the road was insignificant.
Two years ago, after 3 decades of riding an average of 10 miles per year, if you would have told me that I'd be looking froward to putting on padded bike shorts, jersey and bike shoes then hopping on a $2k road bike to head out for 30, 40, 50 or more mile ride, I'd have thought you insane. But here I am.
You'll most likely find that once you get dressed and start riding, you will quickly forget what you look like to others and just enjoy the ride.