I tried flipping it around the other way on the pinch bolt, however, this was clearly not the correct way to do it. It probably doesn't show that well in the pictures, but when it is flipped around the other way, it only pulls the cage down, not to the side. Hence with that arrangement, it is not possible to move the cage at all with the shifter.
However, it seem to have found the problem.
The problem appears to be that my bike just had installed mudguards, and the mudguard on the rear wheel prevents full movement of the derailleur. I didn't notice this at first, but it pushes against the plastic and deforms the mudguard slightly, but this also somewhat limits its movement. After cutting out a piece of the plastic to allow it full movement, it works much better, although not perfectly.
Thanks for all the help guys.