Reasons not to:
Wicking t shirts are lighter, pack smaller, dry faster and are more versatile for off the bike around town type stuff.
i find that I don't generally need the venting of a zipper or the pockets. I have plenty of storage on my bike. Jerseys are great when I'm on a go fast ride, but don't really make sense for touring.
I do wear mountain bike shorts and shoes. Zoic shorts and pearl Izumi or shimano shoes are favorites of mine, along with Patagonia and smart wool t shirts.
Pretty similar to me - I own road jerseys but I don't usually bother with them when touring. I also don't really care for bibs when I tour as I find them inconvenient and they don't pack up as small as shorts. I'll bring both road & mountain bike shorts, and I bring some chamois liners for the mountain bike shorts so I've always got a clean chamois to put on no matter what I'm wearing.
The mountain bike shorts are kind of nice because they double as street clothes.