Femoral Neck break a few years ago in a crit that was in mid June, and I was lucky, because the Doctor came in Sunday night to fix me up/drill home 3 deck screws, and necrosis was prevented. I was spinning on the trainer in six weeks, started riding outside when the x-rays and the Doctor said it looked good, and went back to commuting on the bike. Soon, weekends were getting back to normal, and Brevets called my name again. I pulled off a 6.5 hour 200k in mid September, and honestly, it took me longer to walk without a limp than it did to be happy on a bike. I was 55 at the time.
Funny. You talk about not believing it was broken. Neither did I, because the pain really wasn't in my hip, it seemed lower. Had some pals fetch my bike, and help me to my truck. Getting out of the kit was tough, but once settled into the truck, driving was fine. Of course, I drove right by the hospital on the way home. Once I decided something was wrong, I let my wife know we may have to head to the ER. She fed me a tuna sandwich, because "when you go to the hospital, you usually have to wait a long time". They were rather concerned about the sandwich when I was going under, but it worked out OK.
You're going to be fine, simply because being in good shape is a huge benefit to overall recovery.