For me, the worst part about riding in the rain is when my brake pads pick up a shard of metal, and I hear it grate against the rim. My bikes are too nice to put up with this.
Fortunately, I really don't have to ride in the rain. A good ten months out of the year, we have plenty of dry days. If I still worked a 9-5 job where I couldn't pick and choose riding days based on weather, I would feel different about it.