You really need to test ride some bikes under conditions you actually plan to ride. If I were you and really planned to ride a fair amount of gravel and dirt regularly, then something like the Vaya, or Awol might be on my short list.
I have been helping out a friend who thought he wanted a racing geometry road bike; until he actually got on a modern one and realized that though those bikes are light and sleek, he just wasn't comfortable. What he thought was racing geometry on his old school bike was actually closer to a modern touring bike. Now he is looking for either a performance hybrid, or possibly a light touring bike not unlike some of the models that have been recommended here.