I am fairly confident I just pushed the pin further into the hole a fraction of a mm and didn't deform anything.
If all else fails, I just bought a $150 Shimano 46/30T GRX crank as a backup. (If this shifts smoother than the White Industries crank, the WI crank will become a very expensive spare part.)