Sometimes the pivot on that side gets tight because of dirt, wear, or loss of lubrication on the bushing. Sometimes a thorough cleaning plus a shot of thin lubricant will loosen it. If that doesn't work, you'll have to loosen the pivot. Generally, these are an Allen bolt in front into a nylock nut in the rear arm. There might be a grub screw underneath, too. So loosen the grub screw, loosen the pivot bolt a whisker, test, and tighten the grub screw. Don't loosen it so much that you can twist the arm out of alignment. Then squirt a drop of light lube into the pivot.
And since you're not using the reflector holder attached to the mounting bolt, lose it. It only gets in the way.