In the last couple of years I had some really weird single bike accidents that dumped me out in the middle of pretty busy roads and drivers always stopped and helped me up. In one case an expensive tire broke a cord and blew off the rim throwing me to the "slow" lane of the main street in Oakland, CA, not exactly the world's friendliest town. I had three cars stop and people helping me up instantly. In another case magically a green pine cone appeared under my wheel a quarter mile from the nearest pine tree. I assume that it had been kicked down the road by passing traffic and a car then clipped it so that it shot under my front wheel since it sure as hell wasn't anywhere in front of me. Again two cars stopped to make sure I was OK.
Though they appear to be passing rapidly and not caring whether you're dead or alive, cars are going places too and as long as you're minding your business they're minding theirs.
There were something like 700 bicycle fatalities last year. There were over 40,000 car fatalities. As a cyclist you can pick your roads and learn the safest way to ride. I have ridden on the most dangerous streets in the San Francisco bay area without ever feeling threatened more than I would if I were driving. And I'm a lot smaller target.
The problem isn't that as a new/returning cyclist you shouldn't feel endangered, it is that new drivers do not.