I used to have a problem something like that. Our bodies are seldom symmetrical because handedness. I fixed my issue very simply by hanging from my hands from a bar as long as possible a few times every week. Monkey bars would be even better. Stretched both shoulders out symmetrically. Plus it's good for you.
And yes, paying attention is everything, including to balance and being relaxed. Sight your top tube against your downtube, stem against your front tire, make sure you're centered over your bike. Drop your shoulders, pull your elbows in parallel to centerline.
__________________
Results matter