Lots of good advice above. Getting a bike fit from a credible fitter (I.e. not just a bike shop but a professional - Google bike fit in your area) will get you back on the road sooner and more comfortably, unless your budget will not allow it. I consider it money well spent rather than trial and error with bike shop people unless they are certified
A racing/road style bike versus an upright style bike are two different animals. A road bike will put you in a more unnatural position than you are used to causing back, neck and shoulder pain unless you approach getting your mileage up gradually. This might be frustrating at first, but ride what you can comfortably and then bag it. Slowly up your mileage as your body gets used to it. Road biking takes core strength to support the bent over position - yes your back needs to be straight and not hunched over. Do core and shoulder strengthening exercises to help support your weight while road biking. Google: bike strengthening exercises. Numbness of the hands is caused by too much pressure or holding on for dear life. Loosen your grip and try moving to the top of the bars, the side of the bars, around the bars since variety is your friend - as well as proper padded cycling gloves. The drag about getting into the 40s and beyond is that our bodies are less forgiving and naturally strong, so the way to get there is through doing strengthening exercises, then cycling wont hurt and will be far more enjoyable.
My qualifications for this advice: Like you, and being middle aged, I got a new bike and launched into cycling with great gusto and enthusiasm. I quickly developed all the symptoms (except the knee) you described and was in great pain. Following the advice I gleaned from this forum, undertook the steps above and after 3 months, the pain gradually diminished to now where I can ride 50-60 miles with very minimal discomfort. Distance cycling can never be described as being completely comfortable unless you are an indestructible youth.
You can get there too. Patience, a bike fit and workouts will get you there. BTDT.
Looking forward to progress reports. Cheers