I don't wear bicycle shorts either. Most of my rides are limited by time to about an hour or maybe hour and a half. So I'm not during endurance-type rides where it may be more of a factor. That said, when we vacation to my folks' house in Utah, where we do bike excursions, we might be out for three hours and cover 50 miles. I still don't wear cycling shorts, but I will say that I wouldn't be comfortable doing that all day. But I don't do it all day, either, so it's mostly not relevant to me.
I think it's easy to want to buy a product in order to extend perceived capability, even if you don't actually use that capability. This is fairly common with vehicles, especially pickup trucks. You might buy a larger truck that can tow 10k pounds, you know, just in case. I think there's a parallel in the cycling world, both in terms of cycling equipment (parts, components, etc) and personal wear (clothing, etc). If you're not comfortable now and think padded shorts may help, that's certainly worth a try. But if you're not uncomfortable with your current clothing choices and your riding is somewhat consistent, regarding both the environment and time you're on the bike, then you might not benefit from different choices in clothing.
For those doing "typical recreational riding" who have posted in this thread, it looks like a somewhat even split between folks who do and who don't wear cycling-specific shorts.