I used to race competitively, and with back-to-back races going on sometimes several days, there was no recovery time. There's nothing like spending 5 hours in the saddle the day before, being too sore to bend over to tie your shoes, and wincing in pain to lift your leg over your bike to begin another 4 hour race. But after 30 minutes on the road the stiffness and pain disappear and you are hammering as hard as the day before.
In training nowadays (my racing days are far behind me) I alternate hard days and easy days. After hard days, a meal immediately after my ride (before my shower if my wife isn't paying attention) aids in recovery, as does a good set of stretches. After especially hard rides, I can visit a sports massage specialist who works nearby (he works the the Japanese Olympic team), he does wonders. On easy days I ride "easy," the lowest gear at a pace which kids and housewives pass me. I just pedal for awhile to smooth out the kinks and get the circulation going.