I have also had quite a few concerns about my "car-lite" experience, particularly these last six months in Calgary.
In Toronto, I do have use of a car, which I drive about 1000 km a year in the city (+ another 3-4000 km driving on average per year to Montreal, the Maritimes, Chicago, Boston, etc various trips over the years).
I drive to work about once a month (round trip - 3km), and to the supermarket maybe once every two weeks (also 3 km - with three other people in the vehicle).
Driving this way, while it does generate pollution (including CO2) is definitely a lot lower than the average emissions for USA/Canada.
If you are concerned about carbon dioxide, there are a lot of things that you can do, and driving less is definitely one of them. I don't think you need to beat yourself up over the idea that you do drive sometimes. Try drying your clothes on a clothesline or drying rack - you'll save the CO2 emissions for a short car trip (plus ca$h), or buy efficient appliances (the fridge is a big power hog).
Ride whenever it suits you and be proud that the bike is your primary (even if it's not your only) means of transportation.
My $0.02
Rajman