When I tour, I wear cycling shorts ... black, padded, spandex "roadie" (I guess) cycling shorts. I do carry something else with me like basketball shorts or capris to wear over them for when I stop at a café or something.
I used to wear jerseys too, but can't be bothered with them. A set of wicking Ts suits me better.
But there's nothing wrong with wearing roadie kit, and I'm not quite sure what "tour specific riding gear" would be.