Get. In. The. Gym.
Lots of (most?) cyclists do nothing for their core and upper body. Your
#1 suspension system for your upper body is your triceps. Your
#1 support system is your core. If you do zero training in these area, then yes... you're supposed to have pain.