Well, if the final leg is "through the middle of town," Just own it if you have to.
To me: "Middle of town" suggests a couple of blocks in the innercity downtown core. City cars don't actually go that fast in dense urban environments. In my experience in Honolulu & Seattle keeping up on your bike isn't the hazardous problem you would think it would be so long as you can avoid being perceived as an impediment. Getting good at 20-25 (or more) mph sprints & being mounted & already moving across the intersection by the time the driver behind you gets rolling helps mitigate any animosity an irritated driver may accrue.
In summary, be a car & do your best to:
A) appear to making an effort, and B) appear competent.
If it calms your nerves, remember this: They don't want to hit you either.