Backpack all the way. As already noted, hands are free when off the bike, always have it with me when shopping don't have to worry about accidentally leaving the bag on the bike when locking up, can ride any of my bikes to work or where ever, not just the one that has the rack, prefer the weight on my body instead of the heavy feeling of a weighted down bike when cornering or pedaling out of the saddle.
Have become so accustomed to the bag weight on the body that it's strange to ride without a backpack, feels wrong.