As you've found it's a compromise. Larger frames for me have been more ..... well I'm not sure how to describe. Sort of like driving a nice big sedan on a trip, so stable they seem to steer themselves. Smaller frames, none the less comfortable when properly fit, but handle like a sports car giving you more control to travel in groups and get around those that might be getting in your way.
I thought I'd ride my older and bigger frames on long open road trips. But I find that so far, I don't. The smaller frame is too fun..... like a little sports car.
Don't assume that your sizing for one particular model bike is the same for other models even when of the same manufacturer. If you fall in the overlap, you just have to trust your instinct and previous experience.