As a driver, sidewalk cyclists annoy me. But if it works for the cyclist, I'll get over it. The guys from the work release facility near my office always ride sidewalks.
There are occasions when I'll do it for a block or two, but if I had to do it regularly I think I'd give up riding.