For the same reason your mtb with slick tires is not a road bike - primarily the handlebar shape which encourages a more aero position and gives many options for hand positions, which can reduce fatigue. But if you go riding on your mtb with your buddies on gravel bikes and you can't keep up, it's probably not the bike's fault.
But, in general, you are correct - the whole 'category' of gravel bikes is just a minor tweak on existing bikes. Before gravel bikes there were cyclocross and touring bikes... gravel bikes are like a CX and touring bike had a baby. If you like to explore roads with loose surfaces, though, most road bikes from ~10 or 15 years ago are completely inadequate due to extremely limited tire clearance.