I love it when the passenger yells something behind the anonymity that the door pillar provides. I usually point at my ear and shrug like I didn't hear what they said. If I catch up to them, I ask politely what they yelled as I didn't catch it. That usually doesn't happen as they'll drive through a wall of fire to get away from a cyclist that catches up to them.
It is pretty rare that I ever get harassed though. Most of my riding is on the same routes or on my morning and afternoon commute where I see the same people daily. If the road is clear, I wave them by and I give them a wave or thumbs up if they give me plenty of room.
Now if someone swerves directly at me, throws something at me or puts me in danger, then they will get an earful.
If my visually impaired wife is on the tandem with me and they put us in danger, then things step up a notch. the last time that happened, the driver of the car decided that it was better to run a red light than deal with the PO'd tandem captain directly behind them at a stop light.
Also cameras help. If police ever get involved and a motorist is being a jerk, ask them if they want to see the video. If they think that they've been caught on video, they back down real quick.