I taught people to drive their high-performance cars, primarily Porsches, on race tracks and road courses for 20+ years, and got that question about changing things on their car over and over from novices who wanted to go faster.
My advice to them holds true for bicycles also. The best way to enjoy cycling more is to do the upgrades in this order, giving yourself time in between each step to get comfortable with the changes you made:
1) Improve your ability & get to know your current bike - get lots of seat time
2) Upgrade things that touch the road (i.e. tires, wheels)
3) Upgrade things that touch you, (i.e. saddle, pedals, shoes, etc) and finally
4) Upgrade bicycle components