I think my objection to drafting is more along the lines of "misery doesn't like company." If I am not riding that well, I wonder how screwed up the person drafting me is. And I have always suspected that the answer is that they are pretty screwed up. And I worry they are going to wreck me. The truth is that a lot of randonneurs suck at group riding. And I don't do it very much, so it makes me feel a bit uncomfortable when someone forces it on me. If I chose to ride in a group, that's totally different. I think this would be more evident if someone has ridden PBP. There are so many zombie riders out there. It can be a little scary.
Although in 2011, I was just plain irritable for much of the ride. There was one guy that was drafting me after Fougeres in 2011 that probably would have worked with me, but I was riding really strongly and just dropped him. It didn't hurt that we had a strong tailwind and I didn't need him much. I feel a little bad about that. In 2011, my lighting was a lot better than what many other people had. I had people following me down hills because of that. I didn't really mind that too much.
I resolved not to be irritable this time, and it worked. For the most part anyway.