OP, unless you can find a group as fit and competitive as yourself, you probably would be well-served to try and stop being 'that guy' who has to win every ride, even the 'social' ones. When I was in a club many years ago I did not appreciate the 'fast guys' taking off and leaving everybody else. This ain't the frickin' Tour, ya know? Recreational riders are concerned about fitness - its the main reason we ride, but most of us are looking for the group to more or less stay together. These rides should not be races. Its fine if you want to race, just make sure that everyone in that group also wants to race.