I've done both...and prefer the panniers. Actually a lot of what you list for pros for backpacks apply to panniers. They do cost, but a quality set will serve you a long time. I use Ortliebs. They are made to easily disconnect from the rack and have a shoulder strap for portability, are waterproof, have compartments...as to capacity, panniers come in different sizes, and two large ones will beat the capacity of a typical day-pack.
Cost is probably the only advantage to a backpack, given that you have one you like.