3speed, time and place and judging the surroundings. If a place is a castle along this route and only tourists going in, the chances of them stealing your panniers is a lot less than on some street in some town or city.
of course it isnt ideal, but it depends on the place and surroundings, so one has to make a judgement call.