Just some encouragement here. I did my first tour about three months after giving up drinking beer and eating little debbies as a hobby, and I was never really fit before. It was a road tour, with two friends who were fit and doing 60 to 80 mile days. All I had time for was ten miles every few days, and those really hurt me bad. I did about 300 miles in four days, and it hurt, bad. Only strength of will kept me upright. But I still think that proves anything is possible, it sounds like you will be fine.
You have gotten good advice here on this thread, I have not much to add, other than when you do it, enjoy yourself and don't worry about anyone else who might be faster than you. Today, I can outride both my friends who waited patiently for me at the top of every hill!