If they're holding pace, what's the problem? If they're slowing down, then yeah, they need to pull off and take a break.
I tend to take longer pulls, just depends on how I'm feeling. I usually get thanked for a good pull, never heard anyone complain. There are also guys that do short pulls, nothing wrong with that either, just do what you're feeling.